I want to scrape a specific value of a website. I am using Beautifulsoup. Ussually I am able to get the right tag, but this time I some struggles to get. Here is a piece of the website's code:
<div>
<h1>Wetgeving in wording</h1>
<p>
Wetten en amvb’s waarvan de inwerkingtredingsdatum nog niet bekend is. Ingedeeld op fase van het totstandkomingsproces.
</p>
<div class="well">
<form action="/Regeling/OverviewInWording" id="searchForm" method="get"><input id="Fase" name="Fase" type="hidden" value="Tweede Kamer" /><input data-val="true" data-val-required="The Pagina field is required." id="Pagina" name="Pagina" type="hidden" value="1" /><input data-val="true" data-val-required="The Paginagrootte field is required." id="Paginagrootte" name="Paginagrootte" type="hidden" value="10" /> <fieldset class="form__element">
<legend class="visually-hidden">Zoeken</legend>
<div class="form__element">
<div class="form__element__label">
<label for="BeleidsOnderwerp">Beleidsonderwerp</label>
</div>
<div class="form__element__inputs">
<select id="BeleidsOnderwerp" name="BeleidsOnderwerp"><option selected="selected" value="">Selecteer beleidsonderwerp...</option>
I want to assign the data 'Tweede Kamer' behind 'value=' in the '<form ...' line to a variable. I am able to get the whole div or form, but not specific 'value'.
legislation_phase = soup.find_all("form", action="/Regeling/OverviewInWording")["value"]
What am I doing wrong? Any general tips on finding the right tags are welcome.