python实现将html表格转换成CSV文件的方法


Posted in Python onJune 28, 2015

本文实例讲述了python实现将html表格转换成CSV文件的方法。分享给大家供大家参考。具体如下:

使用方法:python html2csv.py *.html
这段代码使用了 HTMLParser 模块

#!/usr/bin/python
# -*- coding: iso-8859-1 -*-
# Hello, this program is written in Python - http://python.org
programname = 'html2csv - version 2002-09-20 - http://sebsauvage.net'
import sys, getopt, os.path, glob, HTMLParser, re
try:  import psyco ; psyco.jit() # If present, use psyco to accelerate the program
except: pass
def usage(progname):
  ''' Display program usage. '''
  progname = os.path.split(progname)[1]
  if os.path.splitext(progname)[1] in ['.py','.pyc']: progname = 'python '+progname
  return '''%s
A coarse HTML tables to CSV (Comma-Separated Values) converter.
Syntax  : %s source.html
Arguments : source.html is the HTML file you want to convert to CSV.
      By default, the file will be converted to csv with the same
      name and the csv extension (source.html -> source.csv)
      You can use * and ?.
Examples  : %s mypage.html
      : %s *.html
This program is public domain.
Author : Sebastien SAUVAGE <sebsauvage at sebsauvage dot net>
     http://sebsauvage.net
''' % (programname, progname, progname, progname)
class html2csv(HTMLParser.HTMLParser):
  ''' A basic parser which converts HTML tables into CSV.
    Feed HTML with feed(). Get CSV with getCSV(). (See example below.)
    All tables in HTML will be converted to CSV (in the order they occur
    in the HTML file).
    You can process very large HTML files by feeding this class with chunks
    of html while getting chunks of CSV by calling getCSV().
    Should handle badly formated html (missing <tr>, </tr>, </td>,
    extraneous </td>, </tr>...).
    This parser uses HTMLParser from the HTMLParser module,
    not HTMLParser from the htmllib module.
    Example: parser = html2csv()
         parser.feed( open('mypage.html','rb').read() )
         open('mytables.csv','w+b').write( parser.getCSV() )
    This class is public domain.
    Author: Sébastien SAUVAGE <sebsauvage at sebsauvage dot net>
        http://sebsauvage.net
    Versions:
      2002-09-19 : - First version
      2002-09-20 : - now uses HTMLParser.HTMLParser instead of htmllib.HTMLParser.
            - now parses command-line.
    To do:
      - handle <PRE> tags
      - convert html entities (&name; and &#ref;) to Ascii.
      '''
  def __init__(self):
    HTMLParser.HTMLParser.__init__(self)
    self.CSV = ''   # The CSV data
    self.CSVrow = ''  # The current CSV row beeing constructed from HTML
    self.inTD = 0   # Used to track if we are inside or outside a <TD>...</TD> tag.
    self.inTR = 0   # Used to track if we are inside or outside a <TR>...</TR> tag.
    self.re_multiplespaces = re.compile('\s+') # regular expression used to remove spaces in excess
    self.rowCount = 0 # CSV output line counter.
  def handle_starttag(self, tag, attrs):
    if  tag == 'tr': self.start_tr()
    elif tag == 'td': self.start_td()
  def handle_endtag(self, tag):
    if  tag == 'tr': self.end_tr()
    elif tag == 'td': self.end_td()     
  def start_tr(self):
    if self.inTR: self.end_tr() # <TR> implies </TR>
    self.inTR = 1
  def end_tr(self):
    if self.inTD: self.end_td() # </TR> implies </TD>
    self.inTR = 0      
    if len(self.CSVrow) > 0:
      self.CSV += self.CSVrow[:-1]
      self.CSVrow = ''
    self.CSV += '\n'
    self.rowCount += 1
  def start_td(self):
    if not self.inTR: self.start_tr() # <TD> implies <TR>
    self.CSVrow += '"'
    self.inTD = 1
  def end_td(self):
    if self.inTD:
      self.CSVrow += '",' 
      self.inTD = 0
  def handle_data(self, data):
    if self.inTD:
      self.CSVrow += self.re_multiplespaces.sub(' ',data.replace('\t',' ').replace('\n','').replace('\r','').replace('"','""'))
  def getCSV(self,purge=False):
    ''' Get output CSV.
      If purge is true, getCSV() will return all remaining data,
      even if <td> or <tr> are not properly closed.
      (You would typically call getCSV with purge=True when you do not have
      any more HTML to feed and you suspect dirty HTML (unclosed tags). '''
    if purge and self.inTR: self.end_tr() # This will also end_td and append last CSV row to output CSV.
    dataout = self.CSV[:]
    self.CSV = ''
    return dataout
if __name__ == "__main__":
  try: # Put getopt in place for future usage.
    opts, args = getopt.getopt(sys.argv[1:],None)
  except getopt.GetoptError:
    print usage(sys.argv[0]) # print help information and exit:
    sys.exit(2)
  if len(args) == 0:
    print usage(sys.argv[0]) # print help information and exit:
    sys.exit(2)    
  print programname
  html_files = glob.glob(args[0])
  for htmlfilename in html_files:
    outputfilename = os.path.splitext(htmlfilename)[0]+'.csv'
    parser = html2csv()
    print 'Reading %s, writing %s...' % (htmlfilename, outputfilename)
    try:
      htmlfile = open(htmlfilename, 'rb')
      csvfile = open( outputfilename, 'w+b')
      data = htmlfile.read(8192)
      while data:
        parser.feed( data )
        csvfile.write( parser.getCSV() )
        sys.stdout.write('%d CSV rows written.\r' % parser.rowCount)
        data = htmlfile.read(8192)
      csvfile.write( parser.getCSV(True) )
      csvfile.close()
      htmlfile.close()
    except:
      print 'Error converting %s    ' % htmlfilename
      try:  htmlfile.close()
      except: pass
      try:  csvfile.close()
      except: pass
  print 'All done. '

希望本文所述对大家的Python程序设计有所帮助。

Python 相关文章推荐
Python加pyGame实现的简单拼图游戏实例
May 15 Python
详解Python使用simplejson模块解析JSON的方法
Mar 24 Python
wxpython中自定义事件的实现与使用方法分析
Jul 21 Python
使用Python的Django和layim实现即时通讯的方法
May 25 Python
numpy数组之存取文件的实现示例
May 24 Python
pandas DataFrame 交集并集补集的实现
Jun 24 Python
python3实现猜数字游戏
Dec 07 Python
利用pandas合并多个excel的方法示例
Oct 10 Python
django从后台返回html代码的实例
Mar 11 Python
django 多数据库及分库实现方式
Apr 01 Python
浅谈pytorch中的BN层的注意事项
Jun 23 Python
matplotlib制作雷达图报错ValueError的实现
Jan 05 Python
python实现根据主机名字获得所有ip地址的方法
Jun 28 #Python
python自动zip压缩目录的方法
Jun 28 #Python
python查找指定具有相同内容文件的方法
Jun 28 #Python
python中getaddrinfo()基本用法实例分析
Jun 28 #Python
python实现搜索指定目录下文件及文件内搜索指定关键词的方法
Jun 28 #Python
分析用Python脚本关闭文件操作的机制
Jun 28 #Python
python实现linux下使用xcopy的方法
Jun 28 #Python
You might like
ThinkPHP实现非标准名称数据表快速创建模型的方法
2014/11/29 PHP
php递归调用删除数组空值元素的方法
2015/04/28 PHP
php mysql 封装类实例代码
2016/09/18 PHP
thinkphp5框架API token身份验证功能示例
2019/05/21 PHP
Yii redis集合的基本使用教程
2020/06/14 PHP
javascript初学者常用技巧
2014/09/02 Javascript
详解js的事件代理(委托)
2016/12/22 Javascript
Vue.js双向绑定实现原理详解
2016/12/22 Javascript
详解AngularJS1.6版本中ui-router路由中/#!/的解决方法
2017/05/22 Javascript
以BootStrap Tab为例写一个前端组件
2017/07/25 Javascript
angular.js实现购物车功能
2017/10/23 Javascript
react以create-react-app为基础创建项目
2018/03/14 Javascript
小程序云开发实战小结
2018/10/25 Javascript
jQuery使用bind动态绑定事件无效的处理方法
2018/12/11 jQuery
可能被忽略的一些JavaScript数组方法细节
2019/02/28 Javascript
微信小程序位置授权处理方法
2019/06/13 Javascript
微信小程序表单验证插件WxValidate的二次封装功能(终极版)
2019/09/03 Javascript
vue项目从node8.x升级到12.x后的问题解决
2019/10/25 Javascript
openlayers实现地图测距测面
2020/09/25 Javascript
13个最常用的Python深度学习库介绍
2017/10/28 Python
Python中optparser库用法实例详解
2018/01/26 Python
python使用pycharm环境调用opencv库
2018/02/11 Python
python如何去除字符串中不想要的字符
2020/07/05 Python
在django admin中配置搜索域是一个外键时的处理方法
2020/05/20 Python
python3.4中清屏的处理方法
2020/07/06 Python
Gina Bacconi官网:吉娜贝康尼连衣裙和礼服
2018/04/24 全球购物
美术专业个人自我评价
2014/01/18 职场文书
工商管理专业自荐信
2014/06/03 职场文书
纪念9.18事变演讲稿
2014/09/14 职场文书
村长反四风问题个人对照检查材料
2014/09/21 职场文书
房产转让协议书(2014版)
2014/09/30 职场文书
军事博物馆观后感
2015/06/05 职场文书
被委托人身份证明
2015/08/07 职场文书
2016年社区植树节活动总结
2016/03/16 职场文书
解决Jupyter-notebook不弹出默认浏览器的问题
2021/03/30 Python
Redis Stream类型的使用详解
2021/11/11 Redis