Python爬虫——解析常用三大方式之JsonPath-编程知识

Python爬虫——解析常用三大方式之JsonPath

JsonPath

安装

使用

我们的json数据

基本使用

案例

总结

JsonPath

主要适用于解析一些json的数据

安装

pip install -i https://pypi.tuna.tsinghua.edu.cn/simple/ jsonpath

使用

obj = json.load(open( ' json文件 ' , ' r ' , encoding = ' utf-8 ' ) )ret = jsonpath.jsonpath ( obj , ' jsonpath 语法 ' )

可以看看这个文章：JSONPath-简单入门_jsonpath简单入门-CSDN博客

注意：jsonpath和xpath不一样，jsonpath只能解析本地的文件，无法解析服务器的文件

我们的json数据

{"store": {"book": [{"category": "爽文","author": "发飙的蜗牛","title": "妖神记","price": 8.95},{"category": "战争","author": "超神影业","title": "雄兵连","price": 12.99},{"category": "爽文","author": "唐家三少","title": "斗罗大陆","isbn": "0-553-21311-3","price": 8.99},{"category": "爽文","author": "南派三叔","title": "星辰变","isbn": "0-395-19395-8","price": 22.99}],"bicycle": {"color": "蓝色","price": 19.95}}
}

基本使用

import jsonpath
import jsonobj = json.load(open('20_解析_jsonpath.json', 'r', encoding='utf-8'))
# print(obj)
# 书店所有的书
author_book_list = jsonpath.jsonpath(obj, '$.store.book[*].author')
print(author_book_list)# 所有的作者
author_list = jsonpath.jsonpath(obj, '$..author')
print(author_list)# store的所有元素
store_list = jsonpath.jsonpath(obj, '$.store.*')
print(store_list)# store里所有的price
store_price_list = jsonpath.jsonpath(obj, '$.store..price')
print(store_price_list)# 第三本书
book_3 = jsonpath.jsonpath(obj, '$.store.book[2]')
print(book_3)# 最后一本书
book_last = jsonpath.jsonpath(obj, '$..book[(@.length-1)]')
print(book_last)# 前两本书
book_12_way1 = jsonpath.jsonpath(obj, '$.store.book[0,1]')
book_12_way2 = jsonpath.jsonpath(obj, '$.store.book[:2]')
print(book_12_way1)
print(book_12_way2)# 条件过滤需要在 () 的前面添加 ?
# 过滤出所有包含isbn（版本号）的书
book_isbn = jsonpath.jsonpath(obj, '$..book[?(@.isbn)]')
print(book_isbn)# 哪本书超过了十块
book_moreThan_10 = jsonpath.jsonpath(obj, '$..book[?(@.price>10)]')
print(book_moreThan_10)

案例

获取淘淘票支持的全部城市

案例代码：

import urllib.request
import jsonpath
import jsonurl = 'https://dianying.taobao.com/cityAction.json?activityId&_ksTS=1709386130547_108&jsoncallback=jsonp109&action=cityAction&n_s=new&event_submit_doGetAllRegion=true'headers = {# ':authority': 'dianying.taobao.com',# ':method': 'GET',# ':path': '/cityAction.json?activityId&_ksTS=1709386130547_108&jsoncallback=jsonp109&action=cityAction&n_s=new&event_submit_doGetAllRegion=true',# ':scheme': 'https','Accept': 'text/javascript, application/javascript, application/ecmascript, application/x-ecmascript, */*; q=0.01',# 'Accept-Encoding': 'gzip, deflate, br, zstd','Accept-Language': 'zh-CN,zh;q=0.9','Bx-V': '2.5.11','Cookie': 'cna=ckPSHCe/+WACAW8q8CaeQGRA; t=d1bf17f4ff9b1a4f81050b57987cfd21; cookie2=107a2f127e9443a844b590a9101e6a01; v=0; _tb_token_=3a373550bdf81; xlly_s=1; isg=BNPTBeN8shk43n7NaOUT023XYlf9iGdKzTNYOIXw1fIpBPKmDVtGm_XaPnRqo79C','Referer': 'https://dianying.taobao.com/','Sec-Ch-Ua': '"Chromium";v="122", "Not(A:Brand";v="24", "Google Chrome";v="122"','Sec-Ch-Ua-Mobile': '?0','Sec-Ch-Ua-Platform': '"Windows"','Sec-Fetch-Dest': 'empty','Sec-Fetch-Mode': 'cors','Sec-Fetch-Site': 'same-origin','User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36','X-Requested-With': 'XMLHttpRequest',
}request = urllib.request.Request(url=url, headers=headers)response = urllib.request.urlopen(request)content = response.read().decode('utf-8')# print(content)
# 处理掉jsonp109
content = content.split('(')[1].split(')')[0]# 保存到本地
with open('TaoPiaoPiao_city.json' , 'w', encoding='utf-8') as fp:fp.write(content)obj = json.load(open('TaoPiaoPiao_city.json', 'r', encoding='utf-8'))city_list = jsonpath.jsonpath(obj, '$..regionName')
print(city_list)