OH-v3.0-LTS Camera相机驱动框架(L2)解析5之图像采集

系统 OpenHarmony
一个有效的pipeLine 应该必须包含一个SourceNode(VpssNode)和一个SinkNode(VoNode)。

​想了解更多内容,请访问:​

​51CTO和华为官方合作共建的鸿蒙技术社区​

​https://harmonyos.51cto.com​

应用层代码

mainDemo->CaptureON(STREAM_ID_PREVIEW, CAPTURE_ID_PREVIEW, CAPTURE_PREVIEW)
RetCode Hos3516Demo::CaptureON(const int streamId, const int captureId, CaptureMode mode)
{
CAMERA_LOGD("demo test: CaptureON enter mode == %{public}d", mode);
std::shared_ptr<Camera::CaptureInfo> captureInfo = std::make_shared<Camera::CaptureInfo>();
captureInfo->streamIds_ = {streamId};
captureInfo->captureSetting_ = ability_;
captureInfo->enableShutterCallback_ = false;
int rc = streamOperator_->Capture(captureId, captureInfo, true);
......
CAMERA_LOGD("demo test: CaptureON exit");
return RC_OK;
}

一. StreamOperator::Capture

创建一个CaptureRequest请求,调用AddRequest将request添加到对应的Stream里

//drivers/peripheral/camera/hal/hdi_impl/src/stream_operator/stream_operator.cpp
CamRetCode StreamOperator::Capture(int captureId, const std::shared_ptr<CaptureInfo>& captureInfo, bool isStreaming)
{
...
CaptureSetting setting = captureInfo->captureSetting_;
auto request =
std::make_shared<CaptureRequest>(captureId, captureInfo->streamIds_.size(), setting, captureInfo->enableShutterCallback_, isStreaming);
for (auto id : captureInfo->streamIds_) {
RetCode rc = streamMap_[id]->AddRequest(request);
if (rc != RC_OK) {
return DEVICE_ERROR;
}
}
...
return NO_ERROR;
}

StreamBase::AddRequest() 在第一次请求时先调用StartStream()来启动“流”

//drivers/peripheral/camera/hal/hdi_impl/src/stream_operator/stream_base.cpp
RetCode StreamBase::AddRequest(std::shared_ptr<CaptureRequest>& request)
{
......
if (isFirstRequest) {
......
RetCode rc = StartStream();
if (rc != RC_OK) {
CAMERA_LOGE("start stream [id:%{public}d] failed", streamId_);
return RC_ERROR;
}
request->SetFirstRequest(true);
isFirstRequest = false;
}
{
std::unique_lock<std::mutex> l(wtLock_);
waitingList_.emplace_back(request);
cv_.notify_one();
}
return RC_OK;
}

1、pipeline_->Prepare 和 Start 会分别调用前面创建好的流节点列表中每个Node的init和Start接口

2、创建线程处理HandleRequest()

//drivers/peripheral/camera/hal/hdi_impl/src/stream_operator/stream_base.cpp
RetCode StreamBase::StartStream()
{
...
RetCode rc = pipeline_->Prepare({streamId_});
...
state_ = STREAM_STATE_BUSY;
std::string threadName =
g_avaliableStreamType[static_cast<StreamIntent>(streamType_)] + "#" + std::to_string(streamId_);

handler_ = std::make_unique<std::thread>([this, &threadName] {
prctl(PR_SET_NAME, threadName.c_str());
while (state_ == STREAM_STATE_BUSY) {
HandleRequest();
}
});
rc = pipeline_->Start({streamId_});
...
return RC_OK;
}

StartSteam()在完成了必要的启动流程后会通过HandleRequest()的线程,继续处理当前Request的Process()函数。

CaptureRequest的Process启动的是下面的StramBase的Capture()

1、pipeline_->Config 和 Capture 会分别调用前面创建好的流节点列表中每个Node的Config和Capture接口

2、SendMessage负责向应用层上送start的message

3、最后启动buffer分发函数DeliverBuffer() 开始图像数据的传递。

//drivers/peripheral/camera/hal/hdi_impl/src/stream_operator/stream_base.cpp
RetCode StreamBase::Capture(const std::shared_ptr<CaptureRequest>& request)
{
...
rc = pipeline_->Config({streamId_}, request->GetCaptureSetting());
...
rc = pipeline_->Capture({streamId_}, request->GetCaptureId());
...
if (request->IsFirstOne()) {
if (messenger_ == nullptr) {
CAMERA_LOGE("stream [id:%{public}d] can't send message, messenger_ is null", streamId_);
return RC_ERROR;
}
std::shared_ptr<ICaptureMessage> startMessage = std::make_shared<CaptureStartedMessage>(
streamId_, request->GetCaptureId(), request->GetBeginTime(), request->GetOwnerCount());
messenger_->SendMessage(startMessage);
request->SetFirstRequest(false);
}
// DeliverBuffer must be called after Capture, or this capture request will miss a buffer.
do {
rc = DeliverBuffer();
} while (rc != RC_OK && state_ == STREAM_STATE_BUSY);
return RC_OK;
}

到此Capture的请求就处理完成,剩下就是Stream负责让数据流在Node中轮转并通过Surface图形缓冲区接口将数据传递到应用层。

整个的数据轮转涉及到多个线程,下面会把关键的代码整理出来。细节的部分 有兴趣的同学可以自己深入去看。

二. Buffer的申请和下送 StreamBase::DeliverBuffer()

tunnel_->GetBuffer() 会向生产型Surface 要一个Buffer,同时创建一个CameraBuffer(IBuffer类型)和SurfaceBuffer 对应。

取到的IBuffer添加到bufferPool缓冲池来管理。

RetCode StreamBase::DeliverBuffer()
{
CHECK_IF_PTR_NULL_RETURN_VALUE(tunnel_, RC_ERROR);
CHECK_IF_PTR_NULL_RETURN_VALUE(bufferPool_, RC_ERROR);
std::shared_ptr<IBuffer> buffer = tunnel_->GetBuffer();
CHECK_IF_PTR_NULL_RETURN_VALUE(buffer, RC_ERROR);
buffer->SetEncodeType(streamConfig_.encodeType);
buffer->SetStreamId(streamId_);
bufferPool_->AddBuffer(buffer);
CAMERA_LOGI("stream [id:%{public}d] enqueue buffer index:%{public}d", streamId_, buffer->GetIndex());
return RC_OK;
}

bufferPool里有有效的buffer后,会将IBuffer转换成frameSpec格式,下送给SDKnode->ProvideBuffers(frameSpec)

同样是在Vpss这个Node上实现!

//drivers/peripheral/camera/hal/pipeline_core/nodes/src/source_node/source_node.cpp
void SourceNode::PortHandler::CollectBuffers()
{
CHECK_IF_PTR_NULL_RETURN_VOID(pool);
std::shared_ptr<IBuffer> buffer = pool->AcquireBuffer(-1);
CHECK_IF_PTR_NULL_RETURN_VOID(buffer);
PortFormat format = {};
port->GetFormat(format);
std::shared_ptr<FrameSpec> frameSpec = std::make_shared<FrameSpec>();
frameSpec->bufferPoolId_ = format.bufferPoolId_;
frameSpec->bufferCount_ = format.bufferCount_;
frameSpec->buffer_ = buffer;
auto node = port->GetNode();
CHECK_IF_PTR_NULL_RETURN_VOID(node);
RetCode rc = node->ProvideBuffers(frameSpec);
if (rc == RC_ERROR) {
CAMERA_LOGE("provide buffer failed.");
}
}
//drivers/peripheral/camera/hal/adapter/chipset/hispark_taurus/src/pipeline_core/nodes/vpss_node/vpss_node.cpp
RetCode VpssNode::ProvideBuffers(std::shared_ptr<FrameSpec> frameSpec)
{
if (deviceManager_->SendFrameBuffer(frameSpec) == RC_OK) {
return RC_OK;
}
CAMERA_LOGE("provide buffer failed.");
return RC_ERROR;
}

三. 两个重要的Node回调接口

3.1 VpssNode::SetBufferCallback()

在前面pipeline_->Start()中重要的一个节点是VpssNode (VpssNode is-a SourceNode)

这里分别启动了另外两个线程 负责收集Buffer和分发Buffer

另外要看下SetBufferCallback() 给到Hi3516底层的SDK设置重要的回调接口。接口的最终调用是SourceNode::PortHandler::OnBuffer函数。

//drivers/peripheral/camera/hal/pipeline_core/nodes/src/source_node/source_node.cpp
RetCode SourceNode::Start(const int32_t streamId)
{
...
SetBufferCallback();
...
RetCode rc = handler_[streamId]->StartCollectBuffers();
CHECK_IF_NOT_EQUAL_RETURN_VALUE(rc, RC_OK, RC_ERROR);
rc = handler_[streamId]->StartDistributeBuffers();
CHECK_IF_NOT_EQUAL_RETURN_VALUE(rc, RC_OK, RC_ERROR);
return RC_OK;
}
//drivers/peripheral/camera/hal/adapter/chipset/hispark_taurus/src/pipeline_core/nodes/vpss_node/vpss_node.cpp
void VpssNode::SetBufferCallback()
{
deviceManager_->SetNodeCallBack([&](std::shared_ptr<FrameSpec> frameSpec) {
OnPackBuffer(frameSpec);
});
return;
}

3.2 SinkNode::SetCallBack()

另外一个重要的Node是ViNode(ViNode is-a SinkNode)

这个回调的设置要追溯下前几章的代码

从下面的代码片段可以看到SinkNode的CallBack() 调用的是HandleResult() 这个函数。

//drivers\peripheral\camera\hal\hdi_impl\src\stream_operator\stream_base.cpp
RetCode StreamBase::CommitStream()
{
...
RetCode rc = hostStreamMgr_->CreateHostStream(info, [this](std::shared_ptr<IBuffer> buffer) {
HandleResult(buffer);
return;
});
...
}
std::shared_ptr<Pipeline> StreamPipelineBuilder::Build(const std::shared_ptr<PipelineSpec>& pipelineSpec)
{
...
std::optional<int32_t> typeId = GetTypeId(it.type_, G_STREAM_TABLE_PTR, G_STREAM_TABLE_SIZE);
if (typeId) {
newNode->SetCallBack(hostStreamMgr_->GetBufferCb(it.streamId_));
}
...
}

四. camera图像上送

当camera采集到一帧数据,SDK会通过VpssNode的回调通知Camera HDI层。这个时候就来到了上面VpssNode::SetBufferCallback()设置的

SourceNode::PortHandler::OnBuffer()函数。

OnBuffer通知分发线程将接收到的buffers数据分发给实现了DeliverBuffer()接口的Node。本示例代码中SinkNode的DeliverBuffer()

//drivers/peripheral/camera/hal/pipeline_core/nodes/src/sink_node/sink_node.cpp
void SinkNode::DeliverBuffer(std::shared_ptr<IBuffer>& buffer)
{
cb_(buffer);
return;
}

SinkNode的回调StreamBase::HandleResult()最终会来到StreamBase::OnFrame()调用StreamBase::ReceiveBuffer()这个函数;

ReceiveBuffer先把这个buffer归还给bufferPool缓冲池

然后调用StreamTunnel::PutBuffer

//drivers/peripheral/camera/hal/hdi_impl/src/stream_operator/stream_base.cpp
RetCode StreamBase::ReceiveBuffer(std::shared_ptr<IBuffer>& buffer)
{
CHECK_IF_PTR_NULL_RETURN_VALUE(buffer, RC_ERROR);
CHECK_IF_PTR_NULL_RETURN_VALUE(tunnel_, RC_ERROR);
CHECK_IF_PTR_NULL_RETURN_VALUE(bufferPool_, RC_ERROR);
bufferPool_->ReturnBuffer(buffer);
tunnel_->PutBuffer(buffer);
return RC_OK;
}

StreamTunnel会调用Surface的FlushBuffer接口归还一个生产好的SurfaceBuffer对象并携带一些信息。

到此处上层应用的消费型Surface就可以得到这个图像的数据。

RetCode StreamTunnel::PutBuffer(const std::shared_ptr<IBuffer>& buffer)
{
...
if (buffer->GetBufferStatus() == CAMERA_BUFFER_STATUS_OK) {
int32_t fence = 0;
EsFrmaeInfo esInfo = buffer->GetEsFrameInfo();
if (esInfo.size != -1 && esInfo.timestamp != -1) {
sb->ExtraSet("dataSize", esInfo.size);
sb->ExtraSet("isKeyFrame", esInfo.isKey);
sb->ExtraSet("timeStamp", esInfo.timestamp);
sb->ExtraSet("frameNum", esInfo.frameNum);
}
bufferQueue_->FlushBuffer(sb, fence, flushConfig_);
frameCount_++;
} else {
bufferQueue_->CancelBuffer(sb);
}
...
return RC_OK;
}

五.小结

整个取图上送的代码 涉及到很多个线程之间的协同,代码阅读难度相对较大,需要多点耐心

抓住几个重要的Node来梳理代码。

一个有效的pipeLine 应该必须包含一个SourceNode(VpssNode)和一个SinkNode(VoNode)

本章结束后cameraHDI层的总体框架和工作流程介绍完毕。感谢耐心看完系列章节的同学。

​想了解更多内容,请访问:​

​51CTO和华为官方合作共建的鸿蒙技术社区​

​https://harmonyos.51cto.com​


责任编辑:jianghua 来源: 鸿蒙社区
相关推荐

2022-02-14 13:52:04

OpenHarmor系统鸿蒙

2022-02-08 15:07:45

OpenHarmor鸿蒙操作系统

2022-01-06 16:16:21

鸿蒙HarmonyOS应用

2022-02-21 15:38:57

Openharmon操作系统鸿蒙

2022-02-17 16:47:40

OpenharmonIPC通信鸿蒙

2022-06-22 09:14:23

事件打点HiSysEvent

2022-04-21 11:26:31

鸿蒙操作系统

2021-10-20 19:14:30

缓存CacheCPU

2015-01-20 13:19:52

OpenStack网络层数据链路层

2023-11-27 08:21:49

Camera2API,

2014-11-25 13:28:17

openstackneutronDVR

2023-02-13 15:54:49

2021-11-01 17:31:21

Camera2 相机开发

2022-07-14 19:03:33

IPC服务鸿蒙

2022-07-04 16:41:16

IPC通信HiTrace

2023-01-31 09:12:16

CPU芯片缓存

2023-02-20 08:00:00

2021-10-27 11:29:49

Linux框架内核

2023-10-10 15:33:55

机器学习相似性度量

2022-08-26 14:58:43

区块链比特币架构
点赞
收藏

51CTO技术栈公众号