Python中Cookies导出某站用户数据的方法


Posted in Python onMay 17, 2021

应朋友需要,想将某客户的数据从某站里导出,先去某站搞个账号,建几条数据观察一番,心里有底后开搞。

1.Python环境搭建

之前电脑有安装过PyCharm Community 2019.1,具体安装过程就不写了,先跑个HelloWorld,输出正常后正式开整。

2.利用抓包工具或者Google浏览器调试模式拿到请求参数

Cookies参数如下:

cookies = {    
    'JSESSIONID': 'XXX',
    'phone': 'XXX',    
    'password': 'XXX',    
    'isAuto': '0',    '
    loginAccess': 'XXX'
}

headers请求头信息构造:

headers = {    
'Connection': 'keep-alive',    
'sec-ch-ua': '"Google Chrome";v="89", "Chromium";v="89", ";Not A Brand";v="99"',   
'Accept': 'application/json, text/javascript, */*; q=0.01',    'X-Requested-With': 'XMLHttpRequest',    'sec-ch-ua-mobile': '?0',    
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) 
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 
Safari/537.36',    
'Content-Type': 'application/json',   
'Sec-Fetch-Site': 'same-origin',    
'Sec-Fetch-Mode': 'cors',    
'Sec-Fetch-Dest': 'empty',    
'Referer': 'https://xxx.xxx.xxx',    
'Accept-Language': 'zh-CN,zh;q=0.9',}

请求路径:

params = ( ('method', 'getGoodsList'))

请求参数组装:

data = '{
"pageSize":1000,
"pageNumber":1,
"searchText":"",
"sortOrder":"asc",
"isAdvancedSearch":false}'

pageSize:每页数据数量
pageNumber:页码
searchText:搜索条件
sortOrder:排序

3.利用Requests模拟请求并获取数据

response = requests.post(
   'https://xxx.xxx.xxx', 
    headers=headers,
    params=params, 
    cookies=cookies,
    data=data
)
print(response.text.encode('utf8'))
res = json.loads(response.text)

4.创建Excel表格

t = time.time()
randStr = int(round(t * 1000))
tSheetName = "a_" + str(randStr) + ".xlsx"
workbook = xlsxwriter.Workbook(tSheetName)
worksheet = workbook.add_worksheet()

5.表头及数据组装

cursor = 0
picurl = ''
writeExcel(row=cursor)
for obj in res["rows"]:    
    cursor += 1    
    picurl = ''   
    if obj['ImageKID']:        
        picurl = 'https://xxx.xxx.xxx? imageKid='+obj['ImageKID']    
    writeExcel(row=cursor,Description=obj['Description'], Category=obj['Category'], 		 		  Series=obj['Series'],BaseUnit=obj['BaseUnit'],Qty=obj['Qty'],
    CostPrice=obj['CostPrice'],SalePrice=obj['SalePrice'],                    
   RetailPrice=obj['RetailPrice'],Barcode=obj['Barcode'],
   Remark=obj['Remark'], ImageKID=picurl)

6.将数据写入Excel表格中

def writeExcel(row=0, Description='', Category='', Series='', BaseUnit='', Qty='', CostPrice='', SalePrice='', RetailPrice='', Barcode='', Remark='',ImageKID=''):   
	if row == 0:        
		worksheet.write(row, 0, '名称')        
		worksheet.write(row, 1, '货号')        
		worksheet.write(row, 2, '规格')        
		worksheet.write(row, 3, '单位')        
		worksheet.write(row, 4, '库存')        
		worksheet.write(row, 5, '成本')        
		worksheet.write(row, 6, '批发价')        
		worksheet.write(row, 7, '零售价')       
		worksheet.write(row, 8, '条码')        
		worksheet.write(row, 9, '备注')        
		worksheet.write(row, 10, '图片')        
	else:        
   		 if ImageKID!='':            
        		image_data = io.BytesIO(urllib.urlopen(ImageKID).read())            
        		worksheet.insert_image(row, 10, ImageKID, {'image_data': image_data})        
		worksheet.write(row, 0, Description)        
		worksheet.write(row, 1, Category)        
		worksheet.write(row, 2, Series)       
		worksheet.write(row, 3, BaseUnit)        
		worksheet.write(row, 4, Qty)        
		worksheet.write(row, 5, CostPrice)       
		worksheet.write(row, 6, SalePrice)       
		worksheet.write(row, 7, RetailPrice)       
		worksheet.write(row, 8, Barcode)        
		worksheet.write(row, 9, Remark)        
		worksheet.set_column(10, 10, 23)        
		worksheet.set_row(row, 150)

注意图片路径不存在的情况,否则会执行异常

write方法:

def write(self, row, col, *args):
        """
        Write data to a worksheet cell by calling the appropriate write_*()
        method based on the type of data being passed.

        Args:
            row:   The cell row (zero indexed).
            col:   The cell column (zero indexed).
            *args: Args to pass to sub functions.

        Returns:
             0:    Success.
            -1:    Row or column is out of worksheet bounds.
            other: Return value of called method.

        """
        return self._write(row, col, *args)

通过set_row方法设置表格行高

def set_row(self, row, height=None, cell_format=None, options=None):
        """
        Set the width, and other properties of a row.

        Args:
            row:         Row number (zero-indexed).
            height:      Row height. (optional).
            cell_format: Row cell_format. (optional).
            options:     Dict of options such as hidden, level and collapsed.

        Returns:
            0:  Success.
            -1: Row number is out of worksheet bounds.
		......
        """

通过set_column方法设置图片列宽度:

def set_column(self, first_col, last_col, width=None, cell_format=None,
                   options=None):
        """
        Set the width, and other properties of a single column or a
        range of columns.

        Args:
            first_col:   First column (zero-indexed).
            last_col:    Last column (zero-indexed). Can be same as first_col.
            width:       Column width. (optional).
            cell_format: Column cell_format. (optional).
            options:     Dict of options such as hidden and level.

        Returns:
            0:  Success.
            -1: Column number is out of worksheet bounds.
      ......

        """

通过insert_image插入网络图片:

def insert_image(self, row, col, filename, options=None):
        """
        Insert an image with its top-left corner in a worksheet cell.

        Args:
            row:      The cell row (zero indexed).
            col:      The cell column (zero indexed).
            filename: Path and filename for image in PNG, JPG or BMP format.
            options:  Position, scale, url and data stream of the image.

        Returns:
            0:  Success.
            -1: Row or column is out of worksheet bounds.

        """
        # Check insert (row, col) without storing.
        if self._check_dimensions(row, col, True, True):
            warn('Cannot insert image at (%d, %d).' % (row, col))
            return -1

        if options is None:
            options = {}

        x_offset = options.get('x_offset', 0)
        y_offset = options.get('y_offset', 0)
        x_scale = options.get('x_scale', 1)
        y_scale = options.get('y_scale', 1)
        url = options.get('url', None)
        tip = options.get('tip', None)
        anchor = options.get('object_position', 2)
        image_data = options.get('image_data', None)
        description = options.get('description', None)
        decorative = options.get('decorative', False)

        # For backward compatibility with older parameter name.
        anchor = options.get('positioning', anchor)

        if not image_data and not os.path.exists(filename):
            warn("Image file '%s' not found." % force_unicode(filename))
            return -1

        self.images.append([row, col, filename, x_offset, y_offset,
                            x_scale, y_scale, url, tip, anchor, image_data,
                            description, decorative])
        return 0

注意insert_image(row, colunmNum, ‘xx.png', {‘url': xxx})并不能插入网络图片,只是给本地图片一个url路径

7.关闭表格

workbook.close()

8.附引入的包

# -*- coding: UTF-8 -*-
# 批量获取XX数据
import io
import json 
import requests
import sys
import xlsxwriter
import time
import urllib

9.代码跑起来

Python中Cookies导出某站用户数据的方法

在看下Excel表格中导出的信息

Python中Cookies导出某站用户数据的方法

到此这篇关于Python中Cookies导出某站用户数据的方法的文章就介绍到这了,更多相关Python Cookies导出数据内容请搜索三水点靠木以前的文章或继续浏览下面的相关文章希望大家以后多多支持三水点靠木!

Python 相关文章推荐
有关wxpython pyqt内存占用问题分析
Jun 09 Python
Python读取图片EXIF信息类库介绍和使用实例
Jul 10 Python
Python功能键的读取方法
May 28 Python
Python 爬虫的工具列表大全
Jan 31 Python
Python闭包执行时值的传递方式实例分析
Jun 04 Python
对python中GUI,Label和Button的实例详解
Jun 27 Python
python版百度语音识别功能
Jul 09 Python
python通过安装itchat包实现微信自动回复收到的春节祝福
Jan 19 Python
解决pycharm安装第三方库失败的问题
May 09 Python
解析Tensorflow之MNIST的使用
Jun 30 Python
Python基于template实现字符串替换
Nov 27 Python
python小型的音频操作库mp3Play
Apr 24 Python
Python 高级库15 个让新手爱不释手(推荐)
Python带你从浅入深探究Tuple(基础篇)
May 15 #Python
Python中zipfile压缩包模块的使用
python 制作一个gui界面的翻译工具
pyqt5打包成exe可执行文件的方法
Python 机器学习工具包SKlearn的安装与使用
python process模块的使用简介
May 14 #Python
You might like
PHP5中MVC结构学习
2006/10/09 PHP
php面向对象全攻略 (一) 面向对象基础知识
2009/09/30 PHP
php 输出双引号"与单引号'的方法
2010/05/09 PHP
ThinkPHP模板Switch标签用法示例
2014/06/30 PHP
php基于表单密码验证与HTTP验证用法实例
2015/01/06 PHP
PHP使用pear实现mail发送功能 windows环境下配置pear
2016/04/15 PHP
PHP实现打包zip并下载功能
2018/06/12 PHP
laravel 实现关闭CSRF(全部关闭、部分关闭)
2019/10/21 PHP
IE与Firefox下javascript getyear年份的兼容性写法
2007/12/20 Javascript
IE innerHTML,outerHTML所引起的问题
2009/06/04 Javascript
深入理解JavaScript系列(2) 揭秘命名函数表达式
2012/01/15 Javascript
Egret引擎开发指南之视觉编程
2014/09/03 Javascript
JS获取浏览器语言动态加载JS文件示例代码
2014/10/31 Javascript
JavaScript获取网页表单提交方式的方法
2015/04/02 Javascript
JavaScript时间操作之年月日星期级联操作
2016/01/15 Javascript
jQuery 如何给Carousel插件添加新的功能
2016/04/18 Javascript
vue 文件目录结构详解
2017/11/24 Javascript
使用vue-router设置每个页面的title方法
2018/02/11 Javascript
javascript实现文件拖拽事件
2018/03/29 Javascript
echarts同一页面中四个图表切换的js数据交互方法示例
2018/07/03 Javascript
微信小程序使用setData修改数组中单个对象的方法分析
2018/12/30 Javascript
详解一些适用于Node.js的命名约定
2019/12/08 Javascript
vue 虚拟DOM的原理
2020/10/03 Javascript
[41:13]完美世界DOTA2联赛PWL S2 Forest vs Rebirth 第一场 11.20
2020/11/20 DOTA
python 根据正则表达式提取指定的内容实例详解
2016/12/04 Python
Python startswith()和endswith() 方法原理解析
2020/04/28 Python
音频处理 windows10下python三方库librosa安装教程
2020/06/20 Python
Python监听键盘和鼠标事件的示例代码
2020/11/18 Python
Jupyter Notebook 远程访问配置详解
2021/01/11 Python
计算机系毕业生推荐信
2013/11/06 职场文书
2014年3.15团委活动总结
2014/03/16 职场文书
计生工作先进事迹
2014/08/15 职场文书
余世维讲座观后感
2015/06/11 职场文书
高中议论文(范文2篇)
2019/08/19 职场文书
导游词之上海豫园
2019/10/24 职场文书
muduo TcpServer模块源码分析
2022/04/26 Redis