python 提取文件的小程序


Posted in Python onJuly 29, 2009

以前提取这些文件用的是一同事些的批处理文件;用起来不怎么顺手,刚好最近在学些python,所有就自己动手写了一个python提取文件的小程序;
1、原理
提取文件的原理很简单,就是到一个指定的目录,找出最后修改时间大于给定时间的文件,然后将他们复制到目标目录,目标目录的结构必须和原始目录一致,这样工程人员拿到后就可以直接覆盖整个目录;
2、实现
为了程序的通用,我定义了下面的配置文件
config.xml

<?xml version="1.0" encoding="utf-8"?> 
<config> 
    <srcdir>E:\temp\home\cargill</srcdir> 
    <destdir>E:\temp\dest\cargill</destdir> 
    <notinclude> 
        <dirs> 
            <dir>E:\temp\home\cargill\WEB-INF\lib</dir> 
            <dir>E:\temp\home\cargill\static\cargill\report</dir> 
        </dirs> 
        <files> 
            <file>E:\temp\home\cargill\WEB-INF\classes\myrumba.xml</file> 
            <file>E:\temp\home\cargill\META-INF\context.xml</file> 
        </files> 
    </notinclude> 
    <inittime>2008-10-11 13:15:22</inittime> 
    <rardir>C:\Program Files\WinRAR</rardir> 
</config>

其中
<srcdir>:原始目录,即我们tomcat的发布目录;
<destdir>:文件复制到得目标目录;
<notinclude>:需要忽略的文件夹和文件,具体需要忽略的内容在其子节点中定义,这里不在解释;
<inittime>:这个是初始化需要提取的时间点,在这之后的才会提取,此处需要说明,后来在使用中,我增加了一个功能,就是每次提取完会自动将本次提取时间记录到一个文本文件C_UPGRADETIME.txt中,这就省去每次设置这个值的烦恼,只有C_UPGRADETIME.txt为空或者不存在时,才会用到这个值;
<rardir>:rar压缩程序的地址;
下面是读取配置文件的类:
config.py
''' 
Created on Mar 3, 2009 
@author: alex cheng 
''' 
from xml.dom.minidom import parse, parseString 
import datetime 
import time 
class config(object): 
''' 
config.xml 
''' 
def __init__(self, configfile): 
''' 
configfile:config files 
''' 
dom = parse(configfile) 
self.config_element = dom.getElementsByTagName("config")[0] 
def getSrcDir(self): 
''' 
return the <srcdir> element value of self.config_element 
''' 
srcDir = self.config_element.getElementsByTagName("srcdir")[0] 
return self.getText(srcDir.childNodes) 
def getDestDir(self): 
''' 
return the <destdir> element value of self.config_element 
''' 
destDir = self.config_element.getElementsByTagName("destdir")[0] 
return self.getText(destDir.childNodes) 
def getNotIncludeDirs(self): 
''' 
return a list, it's the <dir> element values of self_config_element 
''' 
notinclude_dirs = self.config_element.getElementsByTagName("dir") 
dirList = [] 
for node in notinclude_dirs: 
dir = self.getText(node.childNodes) 
if dir != '': 
dirList.append(dir) 
return dirList 
def getNotIncludeFiles(self): 
''' 
return a list, it's the <file> element values of self.config_element 
''' 
notinclude_files = self.config_element.getElementsByTagName("file") 
fileList = [] 
for node in notinclude_files: 
file = self.getText(node.childNodes) 
if file != '': 
fileList.append(file) 
return fileList 
def getText(self, nodeList): 
''' 
return the text value of the nodeList node 
''' 
rc = '' 
for node in nodeList: 
if node.nodeType == node.TEXT_NODE: 
rc = rc + node.data 
return rc 
def getInitTime(self): 
''' 
return a datetime object,it's the <inittime> element value of self.config_element 
''' 
initTime = self.config_element.getElementsByTagName("inittime")[0] 
timeStr = self.getText(initTime.childNodes) 
dt = datetime.datetime.strptime(timeStr, "%Y-%m-%d %H:%M:%S") 
fdt = time.mktime(dt.utctimetuple()) 
return fdt 
def getWinRarDir(self): 
''' 
return the value of <rardir> element value 
''' 
rardir = self.config_element.getElementsByTagName('rardir')[0] 
return self.getText(rardir.childNodes) 
if __name__ == '__main__': 
c = config('config.xml') 
home = c.getSrcDir() 
print('home is ', home) 
dest = c.getDestDir() 
print('dest is ', dest) 
dirlist = c.getNotIncludeDirs() 
print('not include directory is:') 
for n in dirlist: 
print(n) 
filelist = c.getNotIncludeFiles() 
print('not include files is:') 
for n in filelist: 
print(n) 
inittime = c.getInitTime() 
print('inittime is', inittime) 
rardir = c.getWinRarDir() 
print(rardir)

下面是程序的主体:
fetchfile.py
''' 
Created on Mar 3, 2009 
@author: alex cheng 
''' 
from config import config 
from os import chdir, listdir, makedirs, system, walk, remove, rmdir, unlink, \ 
removedirs, stat, getcwd 
from os.path import abspath, isfile, isdir, join as join_path, exists 
from shutil import copy2 
from sys import path 
import datetime 
import re 
import time 
def getdestdir(dir): 
''' 
return the dest directory name; 
it's named by date,for example 20090101; if 20090101 has exist the return 20090101(1),if 20090101(1) has exist also, 
then return 20090101(2), and then... 
''' 
today = datetime.datetime.today() 
strtoday = today.strftime('%Y%m%d') 
dr = join_path(dir, strtoday) 
tmp = dr 
index = 0 
while isdir(tmp): 
tmp = dr 
index = index + 1 
tmp = tmp + '(' + '%d' % index + ')' 
return tmp 
def fetchFiles(srcdir, destdir, ignoredirs, ignorefiles, lasttime=time.mktime(datetime.datetime(2000, 1, 1).utctimetuple())): 
''' 
fetch files from srcdir(source directory) to destdir(dest directory) ignore the notcopydires(the ignore directory list) 
and notcopyfiles(the ignore file list), and the file and directory's modify time after the lasttime 
''' 
chdir(srcdir) # change the current directory to the srcdir 
dirs = listdir('.') # get all files and directorys in srcdir, but ignore the "." and ".." 
dirlist = [] # save all directorys in srcdir 
for n in dirs: 
if isdir(n): 
dirlist.append(n) 
for subdir in dirlist: 
exist = False 
for ignoredir in ignoredirs: 
if join_path(srcdir, subdir) == ignoredir: 
exist = True 
break 
if exist: 
continue 
fetchFiles(join_path(srcdir, subdir), join_path(destdir, subdir), ignoredirs, ignorefiles, lasttime) 
copyfiles(srcdir, destdir, ignorefiles, lasttime) 
def copyfiles(srcdir, destdir, ignorefiles, lasttime): 
''' 
copy the files from srcdir(source directory) to destdir(dest directory, if dest directory not exist then create is) 
ignore the notcopyfiles(the ignore file list) and the file's modify time must after lasttime 
''' 
chdir(srcdir) 
files = filter(isfile, listdir('.')) 
for file in files: 
if isdir(file): # ignore the directory 
continue 
lastmodify = stat(file).st_mtime 
if lastmodify < lasttime: 
continue 
exist = False 
for ignorefile in ignorefiles: 
if join_path(srcdir, file) == ignorefile: 
exist = True 
if not exist: 
if isdir(destdir) is False: 
try: 
makedirs(destdir) 
print('success create directory:', destdir) 
except: 
raise Exception('failed create directory: ' + destdir) 
try: 
copy2(file, join_path(destdir, file)) 
print('success copy file from', join_path(srcdir, file), 'to', join_path(destdir, file)) 
except: 
raise Exception('failed copy file from ' + join_path(srcdir, file) + ' to ' + join_path(destdir, file)) def tarfiles(dir, todir, winrardir, tarfilename): 
''' 
tar all files in dir(a directory) to todir(dest directory) and the tar file named tarfilename 
''' 
if isdir(dir) is False: 
print('the directory', dir, 'not exist') 
return 
chdir(dir) 
commond = '\"' + winrardir + '\\rar.exe\" a -r ' + todir + '\\' + tarfilename + ' *.*' 
print(commond) 
if system(commond) == 0: 
print('success tar files') 
else: 
print('failed tar files') 
def removeDir(dir_file, currentdir): 
''' 
delete the dir_file 
''' 
if isdir(currentdir) is False: 
print() 
return 
chdir(currentdir) 
if not exists(dir_file): 
return 
if isdir(dir_file): 
for root, dirs, files in walk(dir_file, topdown=False): 
for name in files: 
remove(join_path(root, name)) 
for name in dirs: 
rmdir(join_path(root, name)) 
rmdir(dir_file) # remove the main dir 
else: 
unlink(dir_file) 
return 
def getlasttime(): 
''' 
get last modify time from txt files 
''' 
try: 
mypath = abspath(path[0]) #get current path 
file = join_path(mypath, 'C_UPGRADETIME.txt') 
if isfile(file) is False: 
return 0 
f = open(join_path(mypath, 'C_UPGRADETIME.txt'), 'r') 
lines = f.readlines() 
if len(lines) == 0: 
return 0 
line = lines[ - 1] 
dt = datetime.datetime.strptime(line, "%Y-%m-%d %H:%M:%S") 
lasttime = time.mktime(dt.utctimetuple()) 
f.close() 
return lasttime 
except: 
print('failed to get last modify time from txt file') 
return 0 
def registtime(): 
nowstr = time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(time.time())) 
nowfloat = time.time() 
mypath = abspath(path[0]) # get current path 
f = open(join_path(mypath, 'C_UPGRADETIME.txt'), 'a') 
f.write('\n' + nowstr) 
f.close() 
def main(): 
c = config('config.xml') 
home = c.getSrcDir() 
dest = c.getDestDir() 
ignoreDirs = c.getNotIncludeDirs() 
ignoreFiles = c.getNotIncludeFiles() 
winRarDir = c.getWinRarDir() 
dest = getdestdir(dest)# get current dest directory 
print ('copy all files to the temp directory ignore last fetch time') 
fetchFiles(home, join_path(dest, 'temp'), ignoreDirs, ignoreFiles) 
print('tar the all files') 
tarfiles(join_path(dest, 'temp'), dest, winRarDir, 'CargillUpdate_ALL.rar') 
print('program sleep 20 seconds to finish the tar thread') 
time.sleep(20) 
print('remove the temp directory...') 
removeDir(join_path(dest, 'temp'), dest) 
print('success remove the temp directory') 
lasttime = getlasttime() # get last modify time from txt files 
if lasttime == 0: 
lasttime = c.getInitTime() 
print ('copy all files to the temp2 directory last modify time after last fetch time') 
fetchFiles(home, join_path(dest, 'temp2'), ignoreDirs, ignoreFiles, lasttime) 
print('tar the all files') 
tarfiles(join_path(dest, 'temp2'), dest, winRarDir, 'CargillUpdate.rar') 
print('program sleep 20 seconds to finish the tar thread') 
time.sleep(20) 
print('remove the temp2 directory...') 
removeDir(join_path(dest, 'temp2'), dest) 
print('success remove the temp2 directory') 
registtime() # regist current time 
if __name__ == '__main__': 
main()
Python 相关文章推荐
python检查字符串是否是正确ISBN的方法
Jul 11 Python
浅谈Python基础之I/O模型
May 11 Python
Python 实现在文件中的每一行添加一个逗号
Apr 29 Python
初探利用Python进行图文识别(OCR)
Feb 26 Python
python正则爬取某段子网站前20页段子(request库)过程解析
Aug 10 Python
Python生成验证码、计算具体日期是一年中的第几天实例代码详解
Oct 16 Python
Django中使用MySQL5.5的教程
Dec 18 Python
Numpy 多维数据数组的实现
Jun 18 Python
python利用os模块编写文件复制功能——copy()函数用法
Jul 13 Python
python爬虫使用正则爬取网站的实现
Aug 03 Python
Python中的xlrd模块使用整理
Jun 15 Python
python中redis包操作数据库的教程
Apr 19 Python
Python 文件重命名工具代码
Jul 26 #Python
python 生成目录树及显示文件大小的代码
Jul 23 #Python
python 域名分析工具实现代码
Jul 15 #Python
python 自动提交和抓取网页
Jul 13 #Python
python self,cls,decorator的理解
Jul 13 #Python
python 解析html之BeautifulSoup
Jul 07 #Python
打印出python 当前全局变量和入口参数的所有属性
Jul 01 #Python
You might like
php ss7.5的数据调用 (笔记)
2010/03/08 PHP
ThinkPHP中自定义目录结构的设置方法
2014/08/15 PHP
thinkPHP3.x常量整理(预定义常量/路径常量/系统常量)
2016/05/20 PHP
Yii2创建多界面主题(Theme)的方法
2016/10/08 PHP
对象的类型:本地对象(1)
2006/12/29 Javascript
js滚动条多种样式,推荐
2007/02/05 Javascript
jQuery与其它库冲突的解决方法
2010/06/25 Javascript
jquery miniui 教程 表格控件 合并单元格应用
2012/11/25 Javascript
JavaScript检测浏览器cookie是否已经启动的方法
2015/02/27 Javascript
JavaScript获取表单enctype属性的方法
2015/04/02 Javascript
Bootstrap基本样式学习笔记之表格(2)
2016/12/07 Javascript
纯js实现html转pdf的简单实例(推荐)
2017/02/16 Javascript
JavaScript使用原型和原型链实现对象继承的方法详解
2017/04/05 Javascript
javascript实现文字无缝滚动效果
2017/08/26 Javascript
jQuery实现base64前台加密解密功能详解
2017/08/29 jQuery
详解vue中引入stylus及报错解决方法
2017/09/22 Javascript
clipboard.js在移动端复制失败的解决方法
2018/06/13 Javascript
对 Vue-Router 进行单元测试的方法
2018/11/05 Javascript
JS基于对象的链表实现与使用方法示例
2019/01/31 Javascript
基于vue.js仿淘宝收货地址并设置默认地址的案例分析
2020/08/20 Javascript
vue使用openlayers实现移动点动画
2020/09/24 Javascript
[08:08]DOTA2-DPC中国联赛2月28日Recap集锦
2021/03/11 DOTA
python基于queue和threading实现多线程下载实例
2014/10/08 Python
Python Unittest自动化单元测试框架详解
2018/04/04 Python
Python如何优雅删除字符列表空字符及None元素
2020/06/25 Python
Python3+RIDE+RobotFramework自动化测试框架搭建过程详解
2020/09/23 Python
css3实现背景动态渐变效果
2019/12/10 HTML / CSS
数以千计的折扣工业产品:ESE Direct
2018/05/20 全球购物
国培远程培训感言
2014/03/08 职场文书
小学生期末评语
2014/04/21 职场文书
2014年幼儿园后勤工作总结
2014/11/10 职场文书
文明倡议书
2015/01/19 职场文书
淘宝客服专员岗位职责
2015/04/07 职场文书
酒店人事主管岗位职责
2015/04/11 职场文书
范文之农村基层党建工作报告
2019/10/24 职场文书
Oracle设置DB、监听和EM开机启动的方法
2021/04/25 Oracle