Python – BeautifulSoup is not defined when called in a function

BeautifulSoup is not defined when called in a function… here is a solution to the problem.

BeautifulSoup is not defined when called in a function

When I call BeautifulSoup

() inside my function, my web scraper throws NameError: name 'BeautifulSoup' is not defined, but it works fine when I call it outside the function and pass Soup.

Here is the working code:

from teams.models import *
from bs4 import BeautifulSoup
from django.conf import settings
import requests, os, string

soup = BeautifulSoup(open(os.path.join(settings. BASE_DIR, 'revolver.html')), 'html.parser')

def scrapeTeamPage(soup):
    teamInfo = soup.find('div', 'profile_info')
...
print(scrapeTeamPage(soup))

But when I move the BeautifulSoup call in my function, I get the error.

from teams.models import *
from bs4 import BeautifulSoup
from django.conf import settings
import requests, os, string

def scrapeTeamPage(url):
    soup = BeautifulSoup(open(os.path.join(settings. BASE_DIR, url)), 'html.parser')
    teamInfo = soup.find('div', 'profile_info')

Solution

I’m guessing you made some spelling mistakes with BeautifulSoup, which is case sensitive. If not, use the request in your code:

from teams.models import *
from bs4 import BeautifulSoup
from django.conf import settings
import requests, os, string

def scrapeTeamPage(url):
    res = requests.get(url)
    soup = BeautifulSoup(res.content, 'html.parser')
    teamInfo = soup.find('div', 'profile_info')

Related Problems and Solutions