java解析json复杂数据的两种思路-编程知识

文章目录

一、原始需求
二、简单分析
三、具体实现一
- 1. api接口
- 2. 接口返回
- 3. json 数据解析
- - 1.）引入Jackson库
  - 2.）定义实体
  - 3.）解析json字符串
  - 4.）运行结果
- 4. 过程分析
四、具体实现二
- 1. 核心代码
- 2.运行结果
五、方案比较
六、源码传送

一、原始需求

萌新小明最近新开了CSDN博客，蠢蠢欲动，迫不及待的发表了几篇工作中积累下来的解决问题的涂鸦之作，看着访问量慢慢涨起来，心中暗暗窃喜。现在小明想每天23点记录一下每篇文章的访问量

二、简单分析

对照需求，可以简单分解为如下步骤：

每天23点自动运行任务，实现方式：@Scheduled、cron表达式
获取每篇文章的访问量，实现方式：api接口、数据解析
记录每篇文章的访问量，实现方式：javaBean、JDBC、ORM框架

三、具体实现一

步骤1、3代码实现比较常见，不再详细描述具体实现。
下面我们详细描述步骤2的具体实现

1. api接口

接口地址： https://blog.csdn.net/community/home-api/v1/get-business-list?page={页号}&size={当前页数据条数}&businessType=blog&username={用户名}

2. 接口返回

以下面的接口为例：
https://blog.csdn.net/community/home-api/v1/get-business-list?page=1&size=5&businessType=blog&username=qq_16127313

返回数据如下：

{"code": 200,"message": "success","traceId": "5a64396a-caf3-49ad-a6db-022c55660b75","data": {"list": [{"articleId": 135244727,"title": "java lambda表达式训练题一","description": "Lambda 表达式，也可称为闭包，它是推动 Java 8 发布的最重要新特性。Lambda 允许把函数作为一个方法的参数（函数作为参数传递进方法中）。使用 Lambda 表达式可以使代码变的更加简洁紧凑。","url": "https://blog.csdn.net/qq_16127313/article/details/135244727","type": 1,"top": false,"forcePlan": false,"viewCount": 1019,"commentCount": 0,"editUrl": "https://editor.csdn.net/md?articleId=135244727","postTime": "2023-12-27 18:07:30","diggCount": 7,"formatTime": "2023.12.27","picList": ["https://img-blog.csdnimg.cn/direct/d59c68b950754e879914b5319cd1b53f.png"],"collectCount": 8},{"articleId": 135173565,"title": "二维码初体验 com.google.zxing 实现续 - web api封装","description": "在 二维码初体验 com.google.zxing 实现 我们实现了二维码的生成，但是大部分情况下，二维码的相关功能是作为API接口来提供服务的。我们下面便演示在springboot、Knife4j下封装api接口来实现二维码生成功能。如何使用下面的备份文件恢复成原始的项目代码，请移步查阅：神奇代码恢复工具-over-","url": "https://blog.csdn.net/qq_16127313/article/details/135173565","type": 1,"top": false,"forcePlan": false,"viewCount": 1693,"commentCount": 0,"editUrl": "https://editor.csdn.net/md?articleId=135173565","postTime": "2023-12-23 20:17:11","diggCount": 23,"formatTime": "2023.12.23","picList": ["https://img-blog.csdnimg.cn/direct/f0c994ca789a495a8c8c03d86d626f24.jpeg"],"collectCount": 23},{"articleId": 135167613,"title": "二维码初体验 com.google.zxing 实现","description": "Java 操作二维码的开源项目很多，如 SwetakeQRCode、BarCode4j、Zxing 等，这边以Zxing 为例进行介绍。选择需要生成QR原始文件，支持 “清除空白行及空格” 以减少二维码图片大小。支持输入文本内容，直接生成二维码代码结构QrCodeUI: 完整版本代码SimpleQrCodeUI：简化版本代码如何使用下面的备份文件恢复成原始的项目代码，请移步查阅：神奇代码恢复工具-over-","url": "https://blog.csdn.net/qq_16127313/article/details/135167613","type": 1,"top": false,"forcePlan": false,"viewCount": 1081,"commentCount": 0,"editUrl": "https://editor.csdn.net/md?articleId=135167613","postTime": "2023-12-23 13:52:23","diggCount": 6,"formatTime": "2023.12.23","picList": ["https://img-blog.csdnimg.cn/direct/d3eeac85857543869dce8967c570bdc4.jpeg"],"collectCount": 11},{"articleId": 135135799,"title": "【随笔】MD5加密字符串、文件apache、springframework实现","description": "【代码】【随笔】MD5加密字符串、文件commons-codec、springframework实现。","url": "https://blog.csdn.net/qq_16127313/article/details/135135799","type": 1,"top": false,"forcePlan": false,"viewCount": 1507,"commentCount": 0,"editUrl": "https://editor.csdn.net/md?articleId=135135799","postTime": "2023-12-21 17:29:54","diggCount": 9,"formatTime": "2023.12.21","picList": ["https://img-blog.csdnimg.cn/direct/dc26b7f1c731494f80c8c3b3badfa95d.jpeg"],"collectCount": 9},{"articleId": 135087188,"title": "【随笔】java工程中JSON 字符串格式化输出","description": "json字符串格式化输出fastjson、gson、jackson实现。","url": "https://blog.csdn.net/qq_16127313/article/details/135087188","type": 1,"top": false,"forcePlan": false,"viewCount": 1198,"commentCount": 0,"editUrl": "https://editor.csdn.net/md?articleId=135087188","postTime": "2023-12-19 17:07:40","diggCount": 8,"formatTime": "2023.12.19","picList": ["https://img-blog.csdnimg.cn/direct/058249a1749e4ff5b62e1fcabf516c37.png"],"collectCount": 6}],"total": 71}
}

3. json 数据解析

在Java中，可以使用多种库来解析JSON数据。其中最常用的是fastjson、gson和jackson。
这边我们以jackson为例来说明。

1.）引入Jackson库

<dependency>  <groupId>com.fasterxml.jackson.core</groupId>  <artifactId>jackson-databind</artifactId>  <version>2.13.0</version>  
</dependency>

2.）定义实体

注意这边实体属性一定要跟json数据字段key对应，否则会解析报错

@Data
class BlogData
{private Integer code;private String message;private String traceId;private Record data;
}@Data
class Record
{private List<SubList> list;private Long total;
}@Data
class SubList
{String articleId;String title;String description;String url;Integer type;String top;String forcePlan;Long viewCount;Long commentCount;String editUrl;String postTime;Long diggCount;String formatTime;Object picList;Long collectCount;
}

3.）解析json字符串

        ObjectMapper mapper = new ObjectMapper();String url = "https://blog.csdn.net/community/home-api/v1/get-business-list?page=1&size=5&businessType=blog&username=qq_16127313";String resp = webClient.get().uri(url).acceptCharset(StandardCharsets.UTF_8).accept(MediaType.APPLICATION_JSON).retrieve().bodyToMono(String.class).block();BlogData blogData = mapper.readValue(resp, BlogData.class);log.info("blogData: {} ", blogData);

4.）运行结果

在这里插入图片描述

4. 过程分析

分析整个json数据解析过程，我们发现在实体定义步骤上，花费了我们太多精力和时间，而且一旦json数据key命名有变化或属性名命名不对应，便会打印类似下面的错误信息：

com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "collectCount" (class com.fly.test.restful.json.SubList), not marked as ignorable (15 known properties: "formatTime", "editUrl", "picList", "viewCount", "collectCount1", "postTime", "url", "commentCount", "articleId", "forcePlan", "top", "title", "type", "description", "diggCount"])at [Source: (String)"{"code":200,"message":"success","traceId":"ce6fbb2f-72da-4a9c-9cf4-4e532a942210","data":{"list":[{"articleId":135244727,"title":"java lambda表达式训练题一","description":"Lambda 表达式，也可称为闭包，它是推动 Java 8 发布的最重要新特性。Lambda 允许把函数作为一个方法的参数（函数作为参数传递进方法中）。使用 Lambda 表达式可以使代码变的更加简洁紧凑。","url":"https://blog.csdn.net/qq_16127313/article/details/135244727","type":1,"top":false,"forcePlan":false,"viewCount":1081,"commentCount":0,"editUrl":"https://editor.csdn.net/md?articleId=135244727","postTime":"2023-12-27 18:07:30"[truncated 2457 chars]; line: 1, column: 645] (through reference chain: com.fly.test.restful.json.BlogData["data"]->com.fly.test.restful.json.Record["list"]->java.util.ArrayList[0]->com.fly.test.restful.json.SubList["collectCount"])at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:61)at com.fasterxml.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:840)at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1206)at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1592)at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1570)at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:294)at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:286)at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:245)at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:27)at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:288)at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:288)at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4202)at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3205)at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3173)at com.fly.test.restful.json.ParseJson.test2(ParseJson.java:57)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675)at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:125)at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:132)at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:124)at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:74)at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:104)at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:62)at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:43)at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:35)at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:202)at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:198)at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69)at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)at java.util.ArrayList.forEach(ArrayList.java:1257)at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)at java.util.ArrayList.forEach(ArrayList.java:1257)at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:229)at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:197)at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:211)at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:191)at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:137)at org.eclipse.jdt.internal.junit5.runner.JUnit5TestReference.run(JUnit5TestReference.java:98)at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:40)at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:529)at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:756)at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:452)at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:210)

四、具体实现二

在上面的实现中，通过分析得知，我们关注的信息只有url、viewCount这2个字段值，并不关心其他字段信息。
在【随笔】java工程中JSON 字符串格式化输出中我们已经实现了将json字符串格式化后输出，仔细观察输出信息，会发现简单数据对象（如key、value均为字符串、数字等类型数据）均在同一行，这样我们通过解析输出的行字符串信息，便能得到对应信息。

1. 核心代码

// 调用接口String url = "https://blog.csdn.net/community/home-api/v1/get-business-list?page=1&size=5&businessType=blog&username=qq_16127313";String resp = webClient.get().uri(url).acceptCharset(StandardCharsets.UTF_8).accept(MediaType.APPLICATION_JSON).retrieve().bodyToMono(String.class).block();// 数据格式化-jackson格式化后会在:前后添加空格String pretty = mapper.readTree(resp).toPrettyString();List<String> lines = IOUtils.readLines(new StringReader(pretty));// 获取urlList<String> urls = lines.stream().filter(line -> StringUtils.contains(line, "\"url\"")).map(n -> StringUtils.substringBetween(n, " : \"", "\",")).collect(Collectors.toList());urls.stream().forEach(log::info);// 获取viewCountList<String> viewCounts = lines.stream().filter(line -> StringUtils.contains(line, "\"viewCount\"")).map(n -> StringUtils.substringBetween(n, " : ", ",")).collect(Collectors.toList());viewCounts.stream().forEach(log::info);