I am fairly new to programming and I am currently learning python.I have a web scrapper project using python and beautiful soup library below where I scrap an entire article from a news web page.I manage to scrap the article but I want to query the entire article for specific words and when I try the mongodb query find,I cannot search for any words.Please could someone let me know how to proceed.Thank you for your help.
import pymongo
import requests
url =";
response = requests.get(url)
from bs4 import BeautifulSoup
soup =BeautifulSoup(response.text,'html.parser')
paragraphs = soup.find_all("p")
for paragraph in paragraphs:
print(paragraph.text)
def check(paragraphs):
if len(paragraphs) > 100:
return "extracted fine"
else:
return "An exception occurred"
print(check(paragraphs))
mydict={}
doc = {"paragraphs":mydict}
from pymongo import MongoClient
client= pymongo.MongoClient("mongodb://localhost:27017")
db =client["mydb"]
collection=db["summarizer"]
collection.insert_one(doc)