在项目里快速运行史上最强开源大模型「Llama 3.1」

news/2024/9/18 3:49:55/文章来源:https://www.cnblogs.com/ai2nv/p/18357542
7月份，Meta正式发布迄今为止最强大的开源模型——Llama 3.1，包含了405B、70B和8B三个版本的模型。

 

作为开发者，如何在项目里快速运行LLaMa3.1大模型？

今天，我们结合英智未来自己的LLM推理API平台，给大家介绍一套免费实践流程。

免费使用地址：https://cognihub.baystoneai.com

方式一：不使用流式传输，直接让大模型返回最终结果请求：

# sh
export API_BASE_URL="https://api.baystoneai.com/cognihub/service/66b97dd5fb19cc7397384f42"export API_KEY="Sy0FY5eH1jOaKtcfVvIxN1zsgRWED4IgfoFA"
curl -X POST $API_BASE_URL/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer $API_KEY" -d '{    "model": "llama3.1",    "stream": false,    "messages": [        {"role": "system", "content": "You are a helpful assistant."},        {"role": "user", "content": "英智人工智能智汇API是什么？"}    ]}'

应答：

{  "id": "chatcmpl-754",  "object": "chat.completion",  "created": 1723433933,  "model": "llama3.1",  "system_fingerprint": "fp_ollama",  "choices": [    {      "index": 0,      "message": {        "role": "assistant",        "content": "英智人工智能智汇API（Application Programming Interface）是指一种开放的开发接口，让第三方服务或应用能够访问和调用英智的人工智能技术。通过这些API，开发者可以整合英智的人工智能功能，如自然语言处理、图像识别和情绪分析，来提高自己的产品或服务。\n\n例如，一家移动应用开发商可能会使用英智的语音识别API，让用户能够通过语音说话给应用进行交互。或者，一家教育平台可能会利用英智的情绪分析API，对学生的情绪变化进行实时监测和反馈等应用。\n\n通过API接口，第三方开发者可以使用英智的人工智能能力，为更加强大和有趣的应用或服务提供支持。这也促进了应用和服务之间的快速创新与集成。"      },      "finish_reason": "stop"    }  ],  "usage": {    "prompt_tokens": 31,    "completion_tokens": 186,    "total_tokens": 217  }}
方式二：设置stream（流式传输）请求： 
# sh
export API_BASE_URL="https://api.baystoneai.com/cognihub/service/66b97dd5fb19cc7397384f42"export API_KEY="Sy0FY5eH1jOaKtcfVvIxN1zsgRWED4IgfoFA"
curl -X POST $API_BASE_URL/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer $API_KEY" -d '{    "model": "llama3.1",    "stream": true,    "messages": [        {"role": "system", "content": "You are a helpful assistant."},        {"role": "user", "content": "英智人工智能智汇API是什么？"}    ]}'
应答：
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"英"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"智"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"人"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"工"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"智能"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"智"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"汇"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"API"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"是一"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"种"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"可以"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"为"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"网站"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"、"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"应用"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"程序"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"和"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"其他"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"设备"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"提供"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"话"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"语"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"建议"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"、"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"文"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"本"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"分析"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"等"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"服务"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"的"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"软件"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"接"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"口"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"。"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":"stop"}]}
data: [DONE]

下面，给大家介绍一下「英智LLM推理API平台」，以及平台具有的优势。

 

英智LLM推理API平台

英智LLM推理API是一种基于人工智能大语言模型的推理API服务，它可以帮助开发者简单灵活地使用市面上的所有人工智能大语言模型，为企业定制专属的智能化解决方案。目前已支持包括Llama 3.1、Mistral Large 2、Qwen 2等在内的主流开源大模型的最新版本，同时兼容OpenAI API。

 

 

平台优势

  • 免费注册：申请服务的免费套餐后可获取专属您的“API_KEY”，即可接入本服务来免费使用API。

  • 在线使用：支持在线使用开源大模型。

  • 兼容能力强：提供API调用方式，与OpenAI API接口兼容(只要支持OpenAI API的应用，都可以使用)。

  • 推理速度快：基于NVIDIA高性能GPU算力，推理速度更快。

  • 自然语言处理：通过对大量文本数据进行分析和理解，实现对自然语言的智能处理，如情感分析、文本分类、关键词提取等。

  • 语义理解：通过对自然语言的深入理解，实现对用户意图的准确识别，从而提供更精准的信息服务。

  • 智能问答：通过对大量知识库的学习和理解，实现对用户提问的智能回答，提高用户满意度。

  • 文本生成：通过对大量文本数据的学习，实现对特定主题或场景的文本生成，如新闻稿、产品描述等。

  • 语音识别与合成：通过对语音数据的分析，实现对语音的识别和合成，提高人机交互的自然度和便捷性。

  • 图像识别与处理：通过对图像数据的分析和理解，实现对图像内容的识别和处理，如人脸识别、物体检测等。


免费使用，请访问：https://cognihub.baystoneai.com

加“英智未来”好友，免费体验更多AI产品

 

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.hqwc.cn/news/783405.html

如若内容造成侵权/违法违规/事实不符，请联系编程知识网进行投诉反馈email:809451989@qq.com，一经查实，立即删除！

相关文章

利用Jenkins Pipeline高效部署Kubernetes服务

利用Jenkins Pipeline高效部署Kubernetes服务

什么是 Jenkins Pipeline Jenkins Pipeline是一种持续集成和持续交付（CI/CD）的功能，它允许开发者将复杂的构建、测试和部署流程编码为一系列称为“管道”的自动化步骤。这些步骤以Groovy脚本的形式编写，并且可以在Jenkins中可视化管理。Pipeline提供了代码化和可重用的构建…
阅读更多...
DRM：清华提出无偏差的新类发现与定位新方法 | CVPR 2024

DRM：清华提出无偏差的新类发现与定位新方法 | CVPR 2024

论文分析了现有的新类别发现和定位（NCDL）方法并确定了核心问题：目标检测器往往偏向已知的目标，忽略未知的目标。为了解决这个问题，论文提出了去偏差区域挖掘（DRM）方法，以互补的方式结合类无关RPN和类感知RPN进行目标定位，利用未标记数据的半监督对比学习来改进表征网络…
阅读更多...
城市信息模型：构建未来智慧城市之基底座

城市信息模型：构建未来智慧城市之基底座

在智慧城市的宏大叙事中，城市信息模型（City Information Model, CIM）平台如同城市智能的神经中枢，将数据、空间与技术深度融合，为城市规划、管理、服务、居民生活提供了前所未有的洞察与优化途径。CIM平台的构建不仅是技术的集成，更是智慧城市的灵魂，它以三维可视化、数…
阅读更多...
8.10作业

8.10作业

代码：5秒后跳转马哥教育官网 如下：
阅读更多...
ALV TABLE-COTROL

ALV TABLE-COTROL

1.画屏幕 2.流逻辑CONTROLS gr_9112_control TYPE TABLEVIEW USING SCREEN 9112.PROCESS BEFORE OUTPUT.MODULE status_9112. *处理LOOP从内表读到表控制LOOP WITH CONTROL gr_9112_control.MODULE filltbl1_9112.ENDLOOP.PROCESS AFTER INPUT. *处理LOOP从表控制更新内表LOOP …
阅读更多...
Tornado 龙卷风混币原理

Tornado 龙卷风混币原理

项目背景 Tornado（https://tornado.cash/）是以太坊隐私赛道著名的混币项目，其混币技术主要使用了 zk-SNARK 零知识证明。 1、关于 zk-SNARK 零知识证明的原理可以参见 if(DAO) 之前的文章：https://mirror.xyz/0xd05cFA28Eaf8B4eaFD8Cd86d33c6CeD1a1875417/X3qSOjObTknXQ_iG…
阅读更多...
java浅拷贝BeanUtils.copyProperties引发的RPC异常

java浅拷贝BeanUtils.copyProperties引发的RPC异常

背景 近期参与了一个攻坚项目，前期因为其他流程原因，测试时间已经耽搁了好几天了，本以为已经解决了卡点，后续流程应该顺顺利利的，没想到 人在地铁上，bug从咚咚来~ 没有任何修改的服务接口，抛出异常： java.lang.ClassCastException: java.util.HashMap cannot be cast t…
阅读更多...
RSS 源：在信息洪流中找回你的时间掌控权

RSS 源：在信息洪流中找回你的时间掌控权

简单介绍了 RSS 后，那么关键的一步就是建立好自己的 RSS 源了。200.RSS源管理 简单介绍了 RSS 后，那么关键的一步就是建立好自己的 RSS 源了。 并不是所所有平台都会提供 RSS 源，因此我们也没办法直接去订阅。 目前使用 RSS 的难题之一就是 RSS 源的匮乏，是无数人重新拥抱 …
阅读更多...
P5431 【模板】模意义下的乘法逆元 2

P5431 【模板】模意义下的乘法逆元 2

看到5e6的数据,500ms的时限，\(O(NlogN)\)快速幂直接跑肯定会T掉，那我们就要考虑优化一下式子。 我们令\(s = \prod_{1}^{n}{a[i]}\) ,那我们给第i个式子通分，就为$ \frac{k^i*s/a[i]}{s} $ \(s/a[i]\) 就相当于$ \prod ^{i-1}_{1}{a[i]}* \prod _{i+1}^{n}{a[i]}$ 因此我们只…
阅读更多...
IOS UI自动化脚本

IOS UI自动化脚本

1.安装rethinkdb https://rethinkdb.com/docs/install/osx/ 2.安装bunch包错修复setup.py里面的数据，去掉U 3.imp包变成了importlib需要替换import Imp# moduleFile, pathName, desc = imp.find_module(moduleName, [os.path.dirname(modulePath)])# module = imp.load_module…
阅读更多...
Cartographer的扫描匹配

Cartographer的扫描匹配

cartographer 代码思想解读（1）- 相关匹配cartographer在2016年开源后一直在使用，但是一直未仔细阅读并分析其核心代码结构。目前网上可以找到许多博主对其分析和理解。其cartographer的基本思想可参考他人的 博主博客。本博客主要目的根据其框架思想，将其核心算法进行提取，…
阅读更多...
TapData 信创数据源 | 国产信创数据库 PolarDB MySQL 数据同步指南，加速国产化进程，推进自主创新建设

TapData 信创数据源 | 国产信创数据库 PolarDB MySQL 数据同步指南，加速国产化进程，推进自主创新建设

本专题将以 TapData 正在支持的各国产信创数据源为原点，提供详细的数据库同步、迁移教程，为有需求的用户提供更灵活的工具选择。本文将介绍的 TapData 数据源为——PolarDB。随着国家对自主可控的日益重视，目前在各个行业和区域中面临越来越多的国产化，采用有自主知识产权的…
阅读更多...
推荐文章
最新文章

Copyright @ 2022~2023