RK Camera hal 图像处理

soc:RK3568

system:Android12

今天发现外接的USBCamera用Camera 2API打开显示颠倒,如果在APP 里使用Camera1处理这块接口较少,调整起来比较麻烦

RK Camera hal位置:hardware/interfaces/camera

核心的文件在:

开机会启动:android.hardware.camera.provider@2.4-external-service服务

遍历/dev/videox ,通过V4l2获取摄像头驱动 长 宽 数据格式与帧率,判断当前的摄像头节点是否有效,有效就会告诉CameraServer注册为CameraId,主要代码如下

ExternalCameraDevice.cpp

std::vector<SupportedV4L2Format> ExternalCameraDevice::getCandidateSupportedFormatsLocked(int fd, CroppingType cropType,const std::vector<ExternalCameraConfig::FpsLimitation>& fpsLimits,const std::vector<ExternalCameraConfig::FpsLimitation>& depthFpsLimits,const Size& minStreamSize,bool depthEnabled) {std::vector<SupportedV4L2Format> outFmts;
if (!mSubDevice){// VIDIOC_QUERYCAP get Capabilitystruct v4l2_capability capability;int ret_query = ioctl(fd, VIDIOC_QUERYCAP, &capability);if (ret_query < 0) {ALOGE("%s v4l2 QUERYCAP %s failed: %s", __FUNCTION__, strerror(errno));}struct v4l2_fmtdesc fmtdesc{};fmtdesc.index = 0;if (capability.device_caps & V4L2_CAP_VIDEO_CAPTURE_MPLANE)fmtdesc.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;elsefmtdesc.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;int ret = 0;while (ret == 0) {//获取摄像头格式ret = TEMP_FAILURE_RETRY(ioctl(fd, VIDIOC_ENUM_FMT, &fmtdesc));ALOGV("index:%d,ret:%d, format:%c%c%c%c", fmtdesc.index, ret,fmtdesc.pixelformat & 0xFF,(fmtdesc.pixelformat >> 8) & 0xFF,(fmtdesc.pixelformat >> 16) & 0xFF,(fmtdesc.pixelformat >> 24) & 0xFF);if (ret == 0 && !(fmtdesc.flags & V4L2_FMT_FLAG_EMULATED)) {auto it = std::find (kSupportedFourCCs.begin(), kSupportedFourCCs.end(), fmtdesc.pixelformat);if (it != kSupportedFourCCs.end()) {// Found supported formatv4l2_frmsizeenum frameSize {.index = 0,.pixel_format = fmtdesc.pixelformat};//获取摄像头SIZEfor (; TEMP_FAILURE_RETRY(ioctl(fd, VIDIOC_ENUM_FRAMESIZES, &frameSize)) == 0;++frameSize.index) {if (frameSize.type == V4L2_FRMSIZE_TYPE_DISCRETE) {ALOGV("index:%d, format:%c%c%c%c, w %d, h %d", frameSize.index,fmtdesc.pixelformat & 0xFF,(fmtdesc.pixelformat >> 8) & 0xFF,(fmtdesc.pixelformat >> 16) & 0xFF,(fmtdesc.pixelformat >> 24) & 0xFF,frameSize.discrete.width, frameSize.discrete.height);// Disregard h > w formats so all aspect ratio (h/w) <= 1.0// This will simplify the crop/scaling logic down the roadif (frameSize.discrete.height > frameSize.discrete.width) {continue;}// Discard all formats which is smaller than minStreamSizeif (frameSize.discrete.width < minStreamSize.width|| frameSize.discrete.height < minStreamSize.height) {continue;}SupportedV4L2Format format {.width = frameSize.discrete.width,.height = frameSize.discrete.height,.fourcc = fmtdesc.pixelformat};//获取对于的摄像头参数if (format.fourcc == V4L2_PIX_FMT_Z16 && depthEnabled) {updateFpsBounds(fd, cropType, depthFpsLimits, format, outFmts);} else {updateFpsBounds(fd, cropType, fpsLimits, format, outFmts);}}}
#ifdef HDMI_ENABLEif(strstr((const char*)capability.driver,"hdmi")){ALOGE("driver.find :%s",capability.driver);struct v4l2_dv_timings timings;if(TEMP_FAILURE_RETRY(ioctl(fd, VIDIOC_SUBDEV_QUERY_DV_TIMINGS, &timings)) == 0){char fmtDesc[5]{0};sprintf(fmtDesc,"%c%c%c%c",fmtdesc.pixelformat & 0xFF,(fmtdesc.pixelformat >> 8) & 0xFF,(fmtdesc.pixelformat >> 16) & 0xFF,(fmtdesc.pixelformat >> 24) & 0xFF);ALOGV("hdmi index:%d,ret:%d, format:%s", fmtdesc.index, ret,fmtDesc);ALOGE("%s, hdmi I:%d, wxh:%dx%d", __func__,timings.bt.interlaced, timings.bt.width, timings.bt.height);ALOGV("add hdmi index:%d,ret:%d, format:%c%c%c%c", fmtdesc.index, ret,fmtdesc.pixelformat & 0xFF,(fmtdesc.pixelformat >> 8) & 0xFF,(fmtdesc.pixelformat >> 16) & 0xFF,(fmtdesc.pixelformat >> 24) & 0xFF);SupportedV4L2Format formatGet {.width = timings.bt.width,.height = timings.bt.height,.fourcc = fmtdesc.pixelformat};updateFpsBounds(fd, cropType, fpsLimits, formatGet, outFmts);SupportedV4L2Format format_640x360 {.width = 640,.height = 360,.fourcc = fmtdesc.pixelformat};updateFpsBounds(fd, cropType, fpsLimits, format_640x360, outFmts);SupportedV4L2Format format_1920x1080 {.width = 1920,.height = 1080,.fourcc = fmtdesc.pixelformat};updateFpsBounds(fd, cropType, fpsLimits, format_1920x1080, outFmts);}}
#endif}}fmtdesc.index++;}trimSupportedFormats(cropType, &outFmts);}

上面正常跑入,就可以通过dumpsys  media.camera | grep map 获取到支持的摄像头

rk3588_s:/ $ dumpsys media.camera | grep mapDevice 0 maps to "0"Device 1 maps to "1"Device 2 maps to "112"Device 3 maps to "201"

之后Camera 2 API 通过open 会调到CameraServer最终进到ExternalCameraDevice::open

1.openCamera

Return<void> ExternalCameraDevice::open(const sp<ICameraDeviceCallback>& callback, ICameraDevice::open_cb _hidl_cb) {Status status = Status::OK;sp<ExternalCameraDeviceSession> session = nullptr;if (callback == nullptr) {ALOGE("%s: cannot open camera %s. callback is null!",__FUNCTION__, mCameraId.c_str());_hidl_cb(Status::ILLEGAL_ARGUMENT, nullptr);return Void();}//获取摄像头参数if (isInitFailed()) {ALOGE("%s: cannot open camera %s. camera init failed!",__FUNCTION__, mCameraId.c_str());_hidl_cb(Status::INTERNAL_ERROR, nullptr);return Void();}mLock.lock();ALOGV("%s: Initializing device for camera %s", __FUNCTION__, mCameraId.c_str());session = mSession.promote();if (session != nullptr && !session->isClosed()) {ALOGE("%s: cannot open an already opened camera!", __FUNCTION__);mLock.unlock();_hidl_cb(Status::CAMERA_IN_USE, nullptr);return Void();}//打开摄像头unique_fd fd(::open(mDevicePath.c_str(), O_RDWR));
#ifdef SUBDEVICE_ENABLEif(!mSubDevice){if (fd.get() < 0) {int numAttempt = 0;do {ALOGW("%s: v4l2 device %s open failed, wait 33ms and try again",__FUNCTION__, mDevicePath.c_str());usleep(OPEN_RETRY_SLEEP_US); // sleep and try againfd.reset(::open(mDevicePath.c_str(), O_RDWR));numAttempt++;} while (fd.get() < 0 && numAttempt <= MAX_RETRY);if (fd.get() < 0) {ALOGE("%s: v4l2 device open %s failed: %s",__FUNCTION__, mDevicePath.c_str(), strerror(errno));mLock.unlock();_hidl_cb(Status::INTERNAL_ERROR, nullptr);return Void();}}}
#elseif (fd.get() < 0) {int numAttempt = 0;do {ALOGW("%s: v4l2 device %s open failed, wait 33ms and try again",__FUNCTION__, mDevicePath.c_str());usleep(OPEN_RETRY_SLEEP_US); // sleep and try againfd.reset(::open(mDevicePath.c_str(), O_RDWR));numAttempt++;} while (fd.get() < 0 && numAttempt <= MAX_RETRY);if (fd.get() < 0) {ALOGE("%s: v4l2 device open %s failed: %s",__FUNCTION__, mDevicePath.c_str(), strerror(errno));mLock.unlock();_hidl_cb(Status::INTERNAL_ERROR, nullptr);return Void();}}
#endif//创建Sessionsession = createSession(callback, mCfg, mSupportedFormats, mCroppingType,mCameraCharacteristics, mCameraId, std::move(fd));if (session == nullptr) {ALOGE("%s: camera device session allocation failed", __FUNCTION__);mLock.unlock();_hidl_cb(Status::INTERNAL_ERROR, nullptr);return Void();}if (session->isInitFailed()) {ALOGE("%s: camera device session init failed", __FUNCTION__);session = nullptr;mLock.unlock();_hidl_cb(Status::INTERNAL_ERROR, nullptr);return Void();}mSession = session;mLock.unlock();_hidl_cb(status, session->getInterface());return Void();
}

Camera framework 调用ExternalCameraDeviceSession::processCaptureResult(std::shared_ptr<HalRequest>& req)通过enqueueV4l2Frame 获取到每一帧数据,


void ExternalCameraDeviceSession::enqueueV4l2Frame(const sp<V4L2Frame>& frame) {ATRACE_CALL();frame->unmap();ATRACE_BEGIN("VIDIOC_QBUF");v4l2_buffer buffer{};if (mCapability.device_caps & V4L2_CAP_VIDEO_CAPTURE_MPLANE)buffer.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;elsebuffer.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;buffer.memory = V4L2_MEMORY_MMAP;if (V4L2_TYPE_IS_MULTIPLANAR(buffer.type)) {buffer.m.planes = planes;buffer.length = PLANES_NUM;}buffer.index = frame->mBufferIndex;
#ifdef SUBDEVICE_ENABLEif(!isSubDevice()){if (TEMP_FAILURE_RETRY(ioctl(mV4l2Fd.get(), VIDIOC_QBUF, &buffer)) < 0) {ALOGE("%s: QBUF index %d fails: %s", __FUNCTION__,frame->mBufferIndex, strerror(errno));return;}}
#elseif (TEMP_FAILURE_RETRY(ioctl(mV4l2Fd.get(), VIDIOC_QBUF, &buffer)) < 0) {ALOGE("%s: QBUF index %d fails: %s", __FUNCTION__,frame->mBufferIndex, strerror(errno));return;}
#endifATRACE_END();{std::lock_guard<std::mutex> lk(mV4l2BufferLock);mNumDequeuedV4l2Buffers--;}mV4L2BufferReturned.notify_one();
}Status ExternalCameraDeviceSession::processCaptureResult(std::shared_ptr<HalRequest>& req) {ATRACE_CALL();// Return V4L2 buffer to V4L2 buffer queuesp<V3_4::implementation::V4L2Frame> v4l2Frame =static_cast<V3_4::implementation::V4L2Frame*>(req->frameIn.get());enqueueV4l2Frame(v4l2Frame);// NotifyShutternotifyShutter(req->frameNumber, req->shutterTs);// Fill output buffershidl_vec<CaptureResult> results;results.resize(1);CaptureResult& result = results[0];result.frameNumber = req->frameNumber;result.partialResult = 1;result.inputBuffer.streamId = -1;result.outputBuffers.resize(req->buffers.size());for (size_t i = 0; i < req->buffers.size(); i++) {result.outputBuffers[i].streamId = req->buffers[i].streamId;result.outputBuffers[i].bufferId = req->buffers[i].bufferId;if (req->buffers[i].fenceTimeout) {result.outputBuffers[i].status = BufferStatus::ERROR;if (req->buffers[i].acquireFence >= 0) {native_handle_t* handle = native_handle_create(/*numFds*/1, /*numInts*/0);handle->data[0] = req->buffers[i].acquireFence;result.outputBuffers[i].releaseFence.setTo(handle, /*shouldOwn*/false);}notifyError(req->frameNumber, req->buffers[i].streamId, ErrorCode::ERROR_BUFFER);} else {result.outputBuffers[i].status = BufferStatus::OK;// TODO: refactorif (req->buffers[i].acquireFence >= 0) {native_handle_t* handle = native_handle_create(/*numFds*/1, /*numInts*/0);handle->data[0] = req->buffers[i].acquireFence;result.outputBuffers[i].releaseFence.setTo(handle, /*shouldOwn*/false);}}}// Fill capture result metadatafillCaptureResult(req->setting, req->shutterTs);const camera_metadata_t *rawResult = req->setting.getAndLock();V3_2::implementation::convertToHidl(rawResult, &result.result);req->setting.unlock(rawResult);// update inflight records{std::lock_guard<std::mutex> lk(mInflightFramesLock);mInflightFrames.erase(req->frameNumber);}// Callback into frameworkinvokeProcessCaptureResultCallback(results, /* tryWriteFmq */true);freeReleaseFences(results);return Status::OK;
}

接下来主要是initialize,通过开启一些OutputThread ,图像处理线程FormatConvertThread


bool ExternalCameraDeviceSession::initialize() {
#ifdef SUBDEVICE_ENABLEif(!isSubDevice()){if (mV4l2Fd.get() < 0) {ALOGE("%s: invalid v4l2 device fd %d!", __FUNCTION__, mV4l2Fd.get());return true;}}
#elseif (mV4l2Fd.get() < 0) {ALOGE("%s: invalid v4l2 device fd %d!", __FUNCTION__, mV4l2Fd.get());return true;}
#endifstruct v4l2_capability capability;
#ifdef SUBDEVICE_ENABLEint ret = -1;if(!isSubDevice()){ioctl(mV4l2Fd.get(), VIDIOC_QUERYCAP, &capability);}
#elseint ret = ioctl(mV4l2Fd.get(), VIDIOC_QUERYCAP, &capability);
#endifstd::string make, model;if (ret < 0) {ALOGW("%s v4l2 QUERYCAP failed", __FUNCTION__);mExifMake = "Generic UVC webcam";mExifModel = "Generic UVC webcam";} else {// capability.card is UTF-8 encodedchar card[32];int j = 0;for (int i = 0; i < 32; i++) {if (capability.card[i] < 128) {card[j++] = capability.card[i];}if (capability.card[i] == '\0') {break;}}if (j == 0 || card[j - 1] != '\0') {mExifMake = "Generic UVC webcam";mExifModel = "Generic UVC webcam";} else {mExifMake = card;mExifModel = card;}}initOutputThread();if (mOutputThread == nullptr) {ALOGE("%s: init OutputThread failed!", __FUNCTION__);return true;}mOutputThread->setExifMakeModel(mExifMake, mExifModel);mFormatConvertThread->createJpegDecoder();status_t status = initDefaultRequests();if (status != OK) {ALOGE("%s: init default requests failed!", __FUNCTION__);return true;}mRequestMetadataQueue = std::make_unique<RequestMetadataQueue>(kMetadataMsgQueueSize, false /* non blocking */);if (!mRequestMetadataQueue->isValid()) {ALOGE("%s: invalid request fmq", __FUNCTION__);return true;}mResultMetadataQueue = std::make_shared<ResultMetadataQueue>(kMetadataMsgQueueSize, false /* non blocking */);if (!mResultMetadataQueue->isValid()) {ALOGE("%s: invalid result fmq", __FUNCTION__);return true;}// TODO: check is PRIORITY_DISPLAY enough?mOutputThread->run("ExtCamOut", PRIORITY_DISPLAY);mFormatConvertThread->run("ExtFmtCvt", PRIORITY_DISPLAY);#ifdef HDMI_ENABLE
#ifdef HDMI_SUBVIDEO_ENABLEsp<rockchip::hardware::hdmi::V1_0::IHdmi> client = rockchip::hardware::hdmi::V1_0::IHdmi::getService();if(client.get()!= nullptr){::android::hardware::hidl_string deviceId;client->getHdmiDeviceId( [&](const ::android::hardware::hidl_string &id){deviceId = id.c_str();});ALOGE("getHdmiDeviceId:%s",deviceId.c_str());if(strstr(deviceId.c_str(), mCameraId.c_str())){ALOGE("HDMI attach SubVideo %s",mCameraId.c_str());if(strlen(ExternalCameraDevice::kSubDevName)>0){sprintf(main_ctx.dev_name,"%s",ExternalCameraDevice::kSubDevName);ALOGE("main_ctx.dev_name:%s",main_ctx.dev_name);}mSubVideoThread = new SubVideoThread(0);mSubVideoThread->run("SubVideo", PRIORITY_DISPLAY);}}
#endif
#endifreturn false;
}

每一帧都会在bool ExternalCameraDeviceSession::OutputThread::threadLoop() 里做格式转换和裁剪

//通过RGA处理每一帧图像,图像显示有问题可以在里面改

bool ExternalCameraDeviceSession::OutputThread::threadLoop() {std::shared_ptr<HalRequest> req;auto parent = mParent.promote();if (parent == nullptr) {ALOGE("%s: session has been disconnected!", __FUNCTION__);return false;}...} else if (req->frameIn->mFourcc == V4L2_PIX_FMT_NV12){int handle_fd = -1, ret;const native_handle_t* tmp_hand = (const native_handle_t*)(*(halBuf.bufPtr));ret = ExCamGralloc4::get_share_fd(tmp_hand, &handle_fd);if (handle_fd == -1) {LOGE("convert tmp_hand to dst_fd error");return -EINVAL;}ALOGV("%s(%d): halBuf handle_fd(%d)", __FUNCTION__, __LINE__, handle_fd);ALOGV("%s(%d) halbuf_wxh(%dx%d) frameNumber(%d)", __FUNCTION__, __LINE__,halBuf.width, halBuf.height, req->frameNumber);unsigned long vir_addr =  reinterpret_cast<unsigned long>(req->inData);//通过RGA处理每一帧图像,图像显示有问题可以在里面改camera2::RgaCropScale::rga_scale_crop(tempFrameWidth, tempFrameHeight, vir_addr,HAL_PIXEL_FORMAT_YCrCb_NV12,handle_fd,halBuf.width, halBuf.height, 100, false, false,(halBuf.format == PixelFormat::YCRCB_420_SP), is16Align,true);} else if (req->frameIn->mFourcc == V4L2_PIX_FMT_NV16){...
}

int RgaCropScale::rga_scale_crop(int src_width, int src_height,unsigned long src_fd, int src_format,unsigned long dst_fd,int dst_width, int dst_height,int zoom_val, bool mirror, bool isNeedCrop,bool isDstNV21, bool is16Align, bool isYuyvFormat)
{int ret = 0;rga_info_t src,dst;int zoom_cropW,zoom_cropH;int ratio = 0;...//我的是图像需要镜像 可以在这里改if (mirror)src.rotation = HAL_TRANSFORM_ROT_90; //HAL_TRANSFORM_ROT_else src.rotation = HAL_TRANSFORM_ROT_180;...
}

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.hqwc.cn/news/453212.html

如若内容造成侵权/违法违规/事实不符,请联系编程知识网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

zabbix监控mariadb数据库

zabbix监控mariadb数据库 1.创建监控用户及授权 [rootchang ~]# mysql -uroot -p123qqq.A MariaDB [(none)]> CREATE USER monitor% IDENTIFIED BY 123qqq.A; MariaDB [(none)]> GRANT REPLICATION CLIENT,PROCESS,SHOW DATABASES,SHOW VIEW ON *.* TO monitor%; Maria…

C语言实现memcpy、memmove库函数

目录 引言一、库函数介绍二、库函数详解三、源码实现1.memcpy源码实现2.memmove源码实现 四、测试1.memcpy函数2.memmove函数 五、源码1.memcpy源码2.memmove源码 六、参考文献 引言 关于memcpy和memmove这两个函数&#xff0c;不论是算法竞赛还是找工作面试笔试&#xff0c;对…

DevOps落地笔记-14|部署流水线:打造一站式部署的关键平台

上一课时我主要介绍了实现自动化测试的范围、流程和结构图&#xff0c;自动化测试是持续集成实践不可或缺的一部分&#xff0c;从而使得软件向高效率和高质量迈进了一大步。持续集成主要关注的是代码是否可以编译成功、是否可以通过单元测试和验收测试等。但持续集成并不能实现…

【C#】Json转资源并加载

Json文件如下 右键修改json文件属性 【代码】读取Json文件内容 string sTemplate string.Empty; Assembly assembly Assembly.GetExecutingAssembly(); string resourceName assembly.GetName().Name.ToString() ".Json.test.json"; using (Stream stream asse…

【Python之Git使用教程001】Git简介与安装

一、简介 Git其实就是一个分布式版本的控制系统&#xff0c;在分布式版本的控制系统&#xff0c;大家都拥有一个完整的版本库&#xff0c;不需要联网也可以提交修改&#xff0c;所以中心服务器就显得不那么重要。由于大家都拥有一个完整的版本库&#xff0c;所有只需要把各自的…

目标检测:2如何生成自己的数据集

目录 1. 数据采集 2. 图像标注 3. 开源已标记数据集 4. 数据集划分 参考&#xff1a; 1. 数据采集 数据采集是深度学习和人工智能任务中至关重要的一步&#xff0c;它为模型提供了必要的训练样本和测试数据。在实际应用中&#xff0c;数据采集的方法多种多样&#xff0c;每…

Python||五城P.M.2.5数据分析与可视化_使用复式柱状图分析各个城市的P.M.2.5月度差异情况(中)

目录 4.上海市空气质量月度差异 5.沈阳市空气质量月度差异 五城P.M.2.5数据分析与可视化_使用复式柱状图分析各个城市的P.M.2.5月度差异情况 4.上海市空气质量月度差异 import numpy as np import pandas as pd import matplotlib.pyplot as plt#读入文件 sh pd.read_csv(./S…

chisel RegInit/UInt/U

val reg RegInit(0.U(8.W)) //ok val reg RegInit(0.UInt(8.W)) //errU 使用在数字 . 后边50.U UInt 使用在IO(new Bundle val a Input(UInt(8.W)) 或者 def counter(max:UInt, a1:UInt) package emptyimport chisel3._ import chisel3.util._class MyCounter extends …

26.云原生ArgoCD高级之ApplicationSet

云原生专栏大纲 文章目录 ApplicationSet介绍ApplicationSet 特性ApplicationSet 安装ApplicationSet 工作原理ApplicationSet 生成器列表类型生成器集群生成器基础使用方法Label Selector 指定集群Values 字段传递额外的参数 git生成器git目录生成参数排除目录git文件生成器矩…

山西电力市场日前价格预测【2024-02-03】

日前价格预测 预测说明&#xff1a; 如上图所示&#xff0c;预测明日&#xff08;2024-02-03&#xff09;山西电力市场全天平均日前电价为442.47元/MWh。其中&#xff0c;最高日前电价为633.60元/MWh&#xff0c;预计出现在09:30。最低日前电价为367.07元/MWh&#xff0c;预计…

计算机设计大赛 深度学习 机器视觉 车位识别车道线检测 - python opencv

0 前言 &#x1f525; 优质竞赛项目系列&#xff0c;今天要分享的是 &#x1f6a9; 深度学习 机器视觉 车位识别车道线检测 该项目较为新颖&#xff0c;适合作为竞赛课题方向&#xff0c;学长非常推荐&#xff01; &#x1f947;学长这里给一个题目综合评分(每项满分5分) …

Java tomcat 使用spring-task,实现定时任务功能

前言 今天接触到一个需求&#xff0c;需要添加一个定时任务功能&#xff0c;第一反应是启动类EnableScheduling、定时任务方法使用Scheduled实现&#xff0c;导入项目后才发现&#xff0c;这个项目是ssm整合框架的tomcat项目&#xff0c;没有启动类&#xff0c; 于是改变了思路…