Python中Cookies导出某站用户数据的方法


Posted in Python onMay 17, 2021

应朋友需要,想将某客户的数据从某站里导出,先去某站搞个账号,建几条数据观察一番,心里有底后开搞。

1.Python环境搭建

之前电脑有安装过PyCharm Community 2019.1,具体安装过程就不写了,先跑个HelloWorld,输出正常后正式开整。

2.利用抓包工具或者Google浏览器调试模式拿到请求参数

Cookies参数如下:

cookies = {    
    'JSESSIONID': 'XXX',
    'phone': 'XXX',    
    'password': 'XXX',    
    'isAuto': '0',    '
    loginAccess': 'XXX'
}

headers请求头信息构造:

headers = {    
'Connection': 'keep-alive',    
'sec-ch-ua': '"Google Chrome";v="89", "Chromium";v="89", ";Not A Brand";v="99"',   
'Accept': 'application/json, text/javascript, */*; q=0.01',    'X-Requested-With': 'XMLHttpRequest',    'sec-ch-ua-mobile': '?0',    
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) 
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 
Safari/537.36',    
'Content-Type': 'application/json',   
'Sec-Fetch-Site': 'same-origin',    
'Sec-Fetch-Mode': 'cors',    
'Sec-Fetch-Dest': 'empty',    
'Referer': 'https://xxx.xxx.xxx',    
'Accept-Language': 'zh-CN,zh;q=0.9',}

请求路径:

params = ( ('method', 'getGoodsList'))

请求参数组装:

data = '{
"pageSize":1000,
"pageNumber":1,
"searchText":"",
"sortOrder":"asc",
"isAdvancedSearch":false}'

pageSize:每页数据数量
pageNumber:页码
searchText:搜索条件
sortOrder:排序

3.利用Requests模拟请求并获取数据

response = requests.post(
   'https://xxx.xxx.xxx', 
    headers=headers,
    params=params, 
    cookies=cookies,
    data=data
)
print(response.text.encode('utf8'))
res = json.loads(response.text)

4.创建Excel表格

t = time.time()
randStr = int(round(t * 1000))
tSheetName = "a_" + str(randStr) + ".xlsx"
workbook = xlsxwriter.Workbook(tSheetName)
worksheet = workbook.add_worksheet()

5.表头及数据组装

cursor = 0
picurl = ''
writeExcel(row=cursor)
for obj in res["rows"]:    
    cursor += 1    
    picurl = ''   
    if obj['ImageKID']:        
        picurl = 'https://xxx.xxx.xxx? imageKid='+obj['ImageKID']    
    writeExcel(row=cursor,Description=obj['Description'], Category=obj['Category'], 		 		  Series=obj['Series'],BaseUnit=obj['BaseUnit'],Qty=obj['Qty'],
    CostPrice=obj['CostPrice'],SalePrice=obj['SalePrice'],                    
   RetailPrice=obj['RetailPrice'],Barcode=obj['Barcode'],
   Remark=obj['Remark'], ImageKID=picurl)

6.将数据写入Excel表格中

def writeExcel(row=0, Description='', Category='', Series='', BaseUnit='', Qty='', CostPrice='', SalePrice='', RetailPrice='', Barcode='', Remark='',ImageKID=''):   
	if row == 0:        
		worksheet.write(row, 0, '名称')        
		worksheet.write(row, 1, '货号')        
		worksheet.write(row, 2, '规格')        
		worksheet.write(row, 3, '单位')        
		worksheet.write(row, 4, '库存')        
		worksheet.write(row, 5, '成本')        
		worksheet.write(row, 6, '批发价')        
		worksheet.write(row, 7, '零售价')       
		worksheet.write(row, 8, '条码')        
		worksheet.write(row, 9, '备注')        
		worksheet.write(row, 10, '图片')        
	else:        
   		 if ImageKID!='':            
        		image_data = io.BytesIO(urllib.urlopen(ImageKID).read())            
        		worksheet.insert_image(row, 10, ImageKID, {'image_data': image_data})        
		worksheet.write(row, 0, Description)        
		worksheet.write(row, 1, Category)        
		worksheet.write(row, 2, Series)       
		worksheet.write(row, 3, BaseUnit)        
		worksheet.write(row, 4, Qty)        
		worksheet.write(row, 5, CostPrice)       
		worksheet.write(row, 6, SalePrice)       
		worksheet.write(row, 7, RetailPrice)       
		worksheet.write(row, 8, Barcode)        
		worksheet.write(row, 9, Remark)        
		worksheet.set_column(10, 10, 23)        
		worksheet.set_row(row, 150)

注意图片路径不存在的情况,否则会执行异常

write方法:

def write(self, row, col, *args):
        """
        Write data to a worksheet cell by calling the appropriate write_*()
        method based on the type of data being passed.

        Args:
            row:   The cell row (zero indexed).
            col:   The cell column (zero indexed).
            *args: Args to pass to sub functions.

        Returns:
             0:    Success.
            -1:    Row or column is out of worksheet bounds.
            other: Return value of called method.

        """
        return self._write(row, col, *args)

通过set_row方法设置表格行高

def set_row(self, row, height=None, cell_format=None, options=None):
        """
        Set the width, and other properties of a row.

        Args:
            row:         Row number (zero-indexed).
            height:      Row height. (optional).
            cell_format: Row cell_format. (optional).
            options:     Dict of options such as hidden, level and collapsed.

        Returns:
            0:  Success.
            -1: Row number is out of worksheet bounds.
		......
        """

通过set_column方法设置图片列宽度:

def set_column(self, first_col, last_col, width=None, cell_format=None,
                   options=None):
        """
        Set the width, and other properties of a single column or a
        range of columns.

        Args:
            first_col:   First column (zero-indexed).
            last_col:    Last column (zero-indexed). Can be same as first_col.
            width:       Column width. (optional).
            cell_format: Column cell_format. (optional).
            options:     Dict of options such as hidden and level.

        Returns:
            0:  Success.
            -1: Column number is out of worksheet bounds.
      ......

        """

通过insert_image插入网络图片:

def insert_image(self, row, col, filename, options=None):
        """
        Insert an image with its top-left corner in a worksheet cell.

        Args:
            row:      The cell row (zero indexed).
            col:      The cell column (zero indexed).
            filename: Path and filename for image in PNG, JPG or BMP format.
            options:  Position, scale, url and data stream of the image.

        Returns:
            0:  Success.
            -1: Row or column is out of worksheet bounds.

        """
        # Check insert (row, col) without storing.
        if self._check_dimensions(row, col, True, True):
            warn('Cannot insert image at (%d, %d).' % (row, col))
            return -1

        if options is None:
            options = {}

        x_offset = options.get('x_offset', 0)
        y_offset = options.get('y_offset', 0)
        x_scale = options.get('x_scale', 1)
        y_scale = options.get('y_scale', 1)
        url = options.get('url', None)
        tip = options.get('tip', None)
        anchor = options.get('object_position', 2)
        image_data = options.get('image_data', None)
        description = options.get('description', None)
        decorative = options.get('decorative', False)

        # For backward compatibility with older parameter name.
        anchor = options.get('positioning', anchor)

        if not image_data and not os.path.exists(filename):
            warn("Image file '%s' not found." % force_unicode(filename))
            return -1

        self.images.append([row, col, filename, x_offset, y_offset,
                            x_scale, y_scale, url, tip, anchor, image_data,
                            description, decorative])
        return 0

注意insert_image(row, colunmNum, ‘xx.png', {‘url': xxx})并不能插入网络图片,只是给本地图片一个url路径

7.关闭表格

workbook.close()

8.附引入的包

# -*- coding: UTF-8 -*-
# 批量获取XX数据
import io
import json 
import requests
import sys
import xlsxwriter
import time
import urllib

9.代码跑起来

Python中Cookies导出某站用户数据的方法

在看下Excel表格中导出的信息

Python中Cookies导出某站用户数据的方法

到此这篇关于Python中Cookies导出某站用户数据的方法的文章就介绍到这了,更多相关Python Cookies导出数据内容请搜索三水点靠木以前的文章或继续浏览下面的相关文章希望大家以后多多支持三水点靠木!

Python 相关文章推荐
python中os模块详解
Oct 14 Python
Python使用win32com实现的模拟浏览器功能示例
Jul 13 Python
Python轻量级ORM框架Peewee访问sqlite数据库的方法详解
Jul 20 Python
Python利用BeautifulSoup解析Html的方法示例
Jul 30 Python
python中使用正则表达式的连接符示例代码
Oct 10 Python
解决python3 json数据包含中文的读写问题
May 10 Python
python多进程实现文件下载传输功能
Jul 28 Python
Python3.7实现中控考勤机自动连接
Aug 28 Python
Django rstful登陆认证并检查session是否过期代码实例
Aug 13 Python
屏蔽Django admin界面添加按钮的操作
Mar 11 Python
Python不支持 i ++ 语法的原因解析
Jul 22 Python
Django中日期时间型字段进行年月日时分秒分组统计
Nov 27 Python
Python 高级库15 个让新手爱不释手(推荐)
Python带你从浅入深探究Tuple(基础篇)
May 15 #Python
Python中zipfile压缩包模块的使用
python 制作一个gui界面的翻译工具
pyqt5打包成exe可执行文件的方法
Python 机器学习工具包SKlearn的安装与使用
python process模块的使用简介
May 14 #Python
You might like
php 文件夹删除、php清除缓存程序
2009/08/25 PHP
Zend的Registry机制的使用说明
2013/05/02 PHP
利用php+mcDropdown实现文件路径可在下拉框选择
2013/08/07 PHP
[原创]保存的js无法执行的解决办法
2007/02/25 Javascript
jQuery UI Dialog控件中的表单无法正常提交的解决方法
2010/12/19 Javascript
jQuery操作Select的Option上下移动及移除添加等等
2013/11/18 Javascript
js获取当前页面路径示例讲解
2014/01/08 Javascript
教你如何使用firebug调试功能了解javascript闭包和this
2015/03/04 Javascript
js实现可旋转的立方体模型
2016/10/16 Javascript
解决JSON.stringify()自动将中文转译成unicode的问题
2018/01/05 Javascript
基于Vue2x的图片预览插件的示例代码
2018/05/14 Javascript
node.js到底要不要加分号浅析
2018/07/11 Javascript
详解微信小程序的 request 封装示例
2018/08/21 Javascript
jQuery实现根据身份证号获取生日、年龄、性别等信息的方法
2019/01/09 jQuery
vue+elementUI 复杂表单的验证、数据提交方案问题
2019/06/24 Javascript
如何用vue-cli3脚手架搭建一个基于ts的基础脚手架的方法
2019/12/12 Javascript
基于JS实现计算24点算法代码实例解析
2020/07/23 Javascript
浅谈JavaScript中等号、双等号、 三等号的区别
2020/08/06 Javascript
[02:31]2014DOTA2国际邀请赛2009专访:干爹表现出乎意料 看好DK杀回决赛
2014/07/20 DOTA
Python使用urllib2获取网络资源实例讲解
2013/12/02 Python
python字符串连接方法分析
2016/04/12 Python
matplotlib savefig 保存图片大小的实例
2018/05/24 Python
Python实现FTP弱口令扫描器的方法示例
2019/01/31 Python
Python3.4学习笔记之 idle 清屏扩展插件用法分析
2019/03/01 Python
Django之模板层的实现代码
2019/09/09 Python
Pytorch训练过程出现nan的解决方式
2020/01/02 Python
python PIL/cv2/base64相互转换实例
2020/01/09 Python
Keras - GPU ID 和显存占用设定步骤
2020/06/22 Python
Python request post上传文件常见要点
2020/11/20 Python
阿迪达斯意大利在线商店:adidas意大利
2016/09/19 全球购物
影子教师研修方案
2014/06/14 职场文书
2014年最新大专生职业生涯规划书范文
2014/09/13 职场文书
公司股东合作协议书
2014/09/14 职场文书
延安红色之旅心得体会
2014/10/07 职场文书
基于Python实现的购物商城管理系统
2021/04/27 Python
Mysql实现主从配置和多主多从配置
2021/06/02 MySQL