python - Reading network switch html table into pandas dataframe results with "Empty DataFrame"

When I use pd.read_html() to load a table into a pandas dataframe and then run print(dataframe) the output starts with "Empty DataFrame".

Also, I'm having trouble accessing the elements of the dataframe.

Here is the html:

<HTML>
<HEAD>

<TITLE>System Information 10.8.129.1</TITLE>

</HEAD>

<BODY>


<br><H4>Port Statistics</H4>



<table BORDER COLS=5 WIDTH="80%">

<tr><th>Slot/Port</th><th>Intf.</th><th>TX Frm.</th><th>TX Oct.</th><th>RX Frm.</th><th>RX Oct.</th><th>RX BC</th><th>RX MC</th><th>CRC Align.</th><th>Unders.</th><th>Overs.</th><th>Frag.</th><th>Jabbers</th><th>Total Coll.</th><th>Late Coll.</th><tr><th>1/1</th><th>1</th><th>66967821</th><th>3429650783</th><th>96815811</th><th>2328791105</th><th>39571627</th><th>3960333</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th></tr>

<tr><th>1/2</th><th>2</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th></tr>

<tr><th>2/1</th><th>9</th><th>17533</th><th>1899146</th><th>13646</th><th>1821221</th><th>416</th><th>34</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th></tr>

<tr><th>2/2</th><th>10</th><th>12941919</th><th>1909511968</th><th>3896084</th><th>1687222693</th><th>415948</th><th>78</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th></tr>



</table>


</BODY>

</HTML>

I try to read extract the dataframe by running:

import os
import pandas as pd

url='C:\\WebFiles\switch_stats.html'

dataframe = pd.read_html(url)

print(dataframe)

The output is:

[Empty DataFrame
Columns: [(Slot/Port, 1/1, 1/2, 2/1, 2/2), (Intf., 1, 2, 9, 10), (TX Frm., 66967821, 0, 17533, 12941919), (TX Oct., 3429650783, 0, 1899146, 1909511968), (RX Frm., 96815811, 0, 13646, 3896084), (RX Oct., 2328791105, 0, 1821221, 1687222693), (RX BC, 39571627, 0, 416, 415948), (RX MC, 3960333, 0, 34, 78), (CRC Align., 0, 0, 0, 0), (Unders., 0, 0, 0, 0), (Overs., 0, 0, 0, 0), (Frag., 0, 0, 0, 0), (Jabbers, 0, 0, 0, 0), (Total Coll., 0, 0, 0, 0), (Late Coll., 0, 0, 0, 0)]
Index: []]

When I use pd.read_html() to load a table into a pandas dataframe and then run print(dataframe) the output starts with "Empty DataFrame".

Also, I'm having trouble accessing the elements of the dataframe.

Here is the html:

<HTML>
<HEAD>

<TITLE>System Information 10.8.129.1</TITLE>

</HEAD>

<BODY>


<br><H4>Port Statistics</H4>



<table BORDER COLS=5 WIDTH="80%">

<tr><th>Slot/Port</th><th>Intf.</th><th>TX Frm.</th><th>TX Oct.</th><th>RX Frm.</th><th>RX Oct.</th><th>RX BC</th><th>RX MC</th><th>CRC Align.</th><th>Unders.</th><th>Overs.</th><th>Frag.</th><th>Jabbers</th><th>Total Coll.</th><th>Late Coll.</th><tr><th>1/1</th><th>1</th><th>66967821</th><th>3429650783</th><th>96815811</th><th>2328791105</th><th>39571627</th><th>3960333</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th></tr>

<tr><th>1/2</th><th>2</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th></tr>

<tr><th>2/1</th><th>9</th><th>17533</th><th>1899146</th><th>13646</th><th>1821221</th><th>416</th><th>34</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th></tr>

<tr><th>2/2</th><th>10</th><th>12941919</th><th>1909511968</th><th>3896084</th><th>1687222693</th><th>415948</th><th>78</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th></tr>



</table>


</BODY>

</HTML>

I try to read extract the dataframe by running:

import os
import pandas as pd

url='C:\\WebFiles\switch_stats.html'

dataframe = pd.read_html(url)

print(dataframe)

The output is:

[Empty DataFrame
Columns: [(Slot/Port, 1/1, 1/2, 2/1, 2/2), (Intf., 1, 2, 9, 10), (TX Frm., 66967821, 0, 17533, 12941919), (TX Oct., 3429650783, 0, 1899146, 1909511968), (RX Frm., 96815811, 0, 13646, 3896084), (RX Oct., 2328791105, 0, 1821221, 1687222693), (RX BC, 39571627, 0, 416, 415948), (RX MC, 3960333, 0, 34, 78), (CRC Align., 0, 0, 0, 0), (Unders., 0, 0, 0, 0), (Overs., 0, 0, 0, 0), (Frag., 0, 0, 0, 0), (Jabbers, 0, 0, 0, 0), (Total Coll., 0, 0, 0, 0), (Late Coll., 0, 0, 0, 0)]
Index: []]

Share Improve this question edited Mar 24 at 17:52 ouroboros1 14.7k7 gold badges48 silver badges58 bronze badges asked Mar 24 at 17:12 DBacker 112 bronze badges

Can you share the URL of the page whose table you are trying to scrape? – Ifeanyi Idiaye Commented Mar 24 at 17:56

Add a comment |

1 Answer 1

Sorted by: Reset to default 1

The issue is that each Table Row (<tr>) contains only Table Header elements (th), even though only the first row should logically have such elements. That is to say: your table should have Table Data Cell elements (td) after the first row.

By default, pd.read_html looks for th elements to determine what it should treat as the header. Hence, you end up with an empty dataframe with MultiIndex column labels. To avoid this, simply set header=0:

Minimal Reproducible Example

import pandas as pd
from io import StringIO

# proper `td` elements from row 1 onwards
correct_table = """
<table>
    <tr><th>col1</th><th>col2</th></tr>
    <tr><td>1</th><td>A</td></tr>
    <tr><td>2</th><td>B</td></tr>
</table>
"""

df1 = pd.read_html(StringIO(correct_table))[0]

# only `th` elements
incorrect_table = """
<table>
    <tr><th>col1</th><th>col2</th></tr>
    <tr><th>1</th><th>A</th></tr>
    <tr><th>2</th><th>B</th></tr>
</table>
"""

df2 = pd.read_html(StringIO(incorrect_table), header=0)[0]

df1.equals(df2)
# True

Output:

   col1 col2
0     1    A
1     2    B

Above, StringIO(...) mimics your url file read. Without header=0, you get:

pd.read_html(StringIO(incorrect_table))[0].columns

MultiIndex([('col1', '1', '2'),
            ('col2', 'A', 'B')],
           )

科技改变生活-雨落星辰 - 所有的伟大,都源于一个勇敢的开始

python - Reading network switch html table into pandas dataframe results with "Empty DataFrame" - Stack Overfl

1 Answer 1

与本文相关的文章

评论列表(0)