BeautifulSoup is not defined when called in a function… here is a solution to the problem.
BeautifulSoup is not defined when called in a function
When I call BeautifulSoup
() inside my function, my web scraper throws NameError: name 'BeautifulSoup' is not defined
, but it works fine when I call it outside the function and pass Soup.
Here is the working code:
from teams.models import *
from bs4 import BeautifulSoup
from django.conf import settings
import requests, os, string
soup = BeautifulSoup(open(os.path.join(settings. BASE_DIR, 'revolver.html')), 'html.parser')
def scrapeTeamPage(soup):
teamInfo = soup.find('div', 'profile_info')
...
print(scrapeTeamPage(soup))
But when I move the BeautifulSoup call in my function, I get the error.
from teams.models import *
from bs4 import BeautifulSoup
from django.conf import settings
import requests, os, string
def scrapeTeamPage(url):
soup = BeautifulSoup(open(os.path.join(settings. BASE_DIR, url)), 'html.parser')
teamInfo = soup.find('div', 'profile_info')
Solution
I’m guessing you made some spelling mistakes with BeautifulSoup, which is case sensitive. If not, use the request in your code:
from teams.models import *
from bs4 import BeautifulSoup
from django.conf import settings
import requests, os, string
def scrapeTeamPage(url):
res = requests.get(url)
soup = BeautifulSoup(res.content, 'html.parser')
teamInfo = soup.find('div', 'profile_info')