Body tracking:

  • Tracking of up to 6 simultaneous users's skeletons, with 25 joints each
  • Users leaning angle, tracking confidence, Body edge clipping, hands states
  • Per Body found/lost events

Face tracking:

  • Location and orientation of up to 6 simultaneous users's faces
  • Face points (left and right eyes, nose and left and right mouth corners) in 3D and 2D (Color and Infrared space)
  • Faces bounding boxes in Color and Infrared frames space
  • Expressions reading (Engaged, Happy, Looking Away, Mouth Moved, Mouth Open and Left and Right Eyes Open/Closed) and if users are Wearing glasses or not
  • Per Face found/lost events

Sensor control:

  • Global bodies/faces tracking events (found/lost)
  • Init/Uninit sensor
  • Get sensor tilt, ground plane normal and sensor height

Remapping:

  • 3D camera location to Color texture (optionally with depth) and to Depth texture
  • Find depth of a Color texture location
  • Depth point to Color point and to 3D location

Frames (textures):

  • Get each frame FOV and dimensions
  • Toggle frames usage individually
  • Sample a pixel value from the Depth frame and find the depth of a Color pixel

Network Replicated: No

Platform: Win64 only

身体追踪:

  • 跟踪多达6个同时存在的用户骨骼，每个骨骼有25个关节
  • 用户倾斜角度，跟踪信心，身体边缘剪辑，手状态
  • 每个尸体发现/丢失事件

人脸追踪:

  • 最多可同时显示6个用户脸部的位置和方向
  • 3D和2D（颜色和红外空间）中的脸点（左右眼，鼻子和左右嘴角）
  • 在彩色和红外帧空间中面对边界框
  • 阅读的表情（投入、快乐、移开视线、嘴巴移动、嘴巴张开、左右眼张开/闭合），以及使用者是否戴眼镜
  • 每张脸找到/丢失的事件

传感器控制:

  • 全球机构/面孔追踪事件（发现/丢失）
  • Init/Uninit传感器
  • 获取传感器倾斜、地平面法线和传感器高度

重新映射:

  • 3d相机位置到颜色纹理（可选深度）和深度纹理
  • 查找颜色纹理位置的深度
  • 深度点到颜色点和3D位置

框架（纹理）:

  • 获取每个帧FOV和尺寸
  • 单独切换帧的使用
  • 从深度帧采样像素值并找到颜色像素的深度

网络复制: 非也。

平台: 仅限Win64










With the Neo Kinect plugin you can use the Kinect v2 sensor advanced capabilities within Unreal Engine, with easy to use Blueprint nodes, all nicely commented, or directly through the C++ methods. And starting with Unreal Engine 5, the plugin is fully compatible with DX12 and all its perks, like ray tracing and Lumen.

Examples and Quick Start doc

Quick Start guide: NeoKinect-QuickStart.pdf

Example Project for Unreal Engine 5: NeoKinectExamples.zip.

  • This example is not yet using the new demo room from UE5, only the new skeleton (the UE4 one is still there as well).

Example Project for Unreal Engine 4: NeoKinectExamples_UE4.zip.

Videos of projects made with this plugin:

https://www.youtube.com/watch?v=RyDmDAZYbAE

https://www.youtube.com/watch?v=x0RH_-xRhFU

https://www.youtube.com/watch?v=59fnWYRQN88

FAQ

If you have a question about a problem with the plugin, maybe it’s been answered here, at the end of the first post, in the FAQ section.

Release Notes

Minor engine version updates are not listed here as changes. Only changes related to functionality and fixes.

Unreal 5.0 release (April 4, 2022)

Front-end (project and Blueprints):

  • Frames are now returned as TextureRenderTarget2D from SetUseFrame. Widgets accessing the frame textures need to be updated as before a cast to a custom texture format was required. That custom format no longer exists as its customizations are no longer needed. As a bonus, that enables the next item in this list.

  • Now compatible with DX12 and all its perks, like Ray Tracing, Lumen and so on!

Code:

  • Fixed a memory leak when enabling frame types.

  • Lots of IWYU improvements, with less included clutter in public headers.

Features Overview

Robust and fast

The plugin was created with performance and usability in mind, so you can track all 6 possible users, their faces and enable all of the Kinect’s frame types (color, depth, infrared etc) at the same time with almost no hit in performance. Sensor polling is made in its own thread. The frames textures (color camera, depth camera, IR and their variations) are TextureRenderTarget2D (UE5+ only. A custom texture format was used on UE4), which is compatible with materials, UMG Widgets and whatnot. There are functions to access the textures pixel values from Blueprint as well.

No need for components

The sensor is unique, no matter how many Actors or Widgets are using it. So, instead of needing to add components, you just call functions like with a function library. That way you can control the device from any Blueprint, including Widgets.

Advanced Remapping

Besides access to the standard Microsoft Kinect API coordinate remapping methods, the plugin also comes with other remapping features that facilitate AR applications, like getting the location of a joint in the Color frame without losing its depth information. Every location and orientation was adapted to Unreal’s coordinate system and Joints transforms are compatible with the Engine’s Mannequin character rig.

Fully production proven

I’ve used Neo Kinect a lot (more than a year) before releasing to the public and removed all bugs found so far, besides making a lot of performance improvements. It was used in applications that go through a whole day without crashing and packages without problems.

与 新Kinect 插件您可以使用虚幻引擎中的Kinect v2传感器高级功能，具有易于使用的蓝图节点，所有注释都很好，或者直接通过C++方法。 从虚幻引擎5开始，该插件完全兼容DX12及其所有功能，如光线追踪和流明。

示例和快速入门文档

快速入门指南: NeoKinect-快速入门.pdf格式

虚幻引擎5的示例项目: NeoKinectExamples。拉链.

  • 这个例子还没有使用UE5的新演示室，只有新的骨架（UE4的骨架也在那里）。

虚幻引擎4的示例项目: NeoKinectExamples_UE4.拉链.

短片 使用此插件制作的项目:

https://www.youtube.com/watch?v=RyDmDAZYbAE

https://www.youtube.com/watch?v=x0RH_-xRhFU

https://www.youtube.com/watch?v=59fnWYRQN88

常见问题

如果你有一个关于插件问题的问题，也许它已经被回答了 这里，在第一篇文章的末尾，在 常见问题 节。

发行通知书

次要引擎版本更新不会在此处作为更改列出。 仅与功能和修复相关的更改。

Unreal5.0发布（2022年4月4日）

前端（项目和蓝图）:

  • 帧现在作为TextureRenderTarget2D从SetUseFrame返回。 访问框架纹理的小部件需要更新，就像之前需要转换为自定义纹理格式一样。 该自定义格式不再存在，因为不再需要其自定义。 作为奖励，这将启用此列表中的下一个项目。

  • 现在兼容DX12和它的所有特权，如光线追踪，流明等！

密码:

  • 修复了启用帧类型时的内存泄漏。

  • 大量的IWYU改进，较少包含公共标题中的混乱。

功能概述

健壮和快速

该插件的创建考虑了性能和可用性，因此您可以跟踪所有6个可能的用户，他们的脸，并启用所有Kinect的框架类型（颜色，深度，红外等）在同一时间几乎没有击中性能。 传感器轮询是在自己的线程中进行的。 帧纹理（彩色相机，深度相机，红外和它们的变化）是TextureRenderTarget2D（仅限UE5+。 在UE4上使用了一个自定义纹理格式，它与材质、UMG小部件和诸如此类的东西兼容。 还有一些函数可以从蓝图中访问纹理像素值。

无需组件

无论有多少Actor或小部件使用它，传感器都是独一无二的。 因此，您只需像使用函数库一样调用函数，而无需添加组件。 这样，您就可以从任何蓝图（包括控件）控制设备。

高级重映射

除了访问标准的Microsoft Kinect API坐标重映射方法外，该插件还附带了其他有助于AR应用程序的重映射功能，例如在不丢失深度信息的情况下获取颜色帧中关节 每个位置和方向都适应了虚幻的坐标系，关节变换与引擎的人体模型角色装备兼容。

完全生产证明

在向公众发布之前，我已经使用了Neo Kinect（一年多），并删除了迄今为止发现的所有错误，除了进行了大量的性能改进。 它被用于一整天都没有崩溃和包没有问题的应用程序。

声明：本站所有资源都是由站长从网络上收集而来，如若本站内容侵犯了原著者的合法权益，可联系站长删除。