It’s been one month I’ve been working on this Intel realsense D435, and still I’m not so sure with a lot of functions and some parameters. Like when to add () and when not, which function need which
parameter to fit in. First I read the
https://buildmedia.readthedocs.org/media/pdf/pyrealsense/dev/pyrealsense.pdf
And the python.cpp in the python wrapper to find the options Actually it didn’t help much, mainly I started with the examples in python wrapper and then read the corresponding codes in the C++ example. It is also because this project is to measure objects in the picture, and in the example there is already one measuring in stream, so translating that code is the first approach. And here
https://pyrealsense.readthedocs.io/en/master/index.html
The starting part of the camera:
With the Intel Realsense Viewer we can get all the possible control options
the picture of IR/Depth/RGB stream
For RGB the highest resolution can be up to 1920*1080
auto exposure sometimes is turned off to reduce frame drop
In my measuring usage I will be using the High density to get the most pixels covered
while determine movement or object can use high accuracy
and the presets can be also set manually
one important feature for me is the hole filling, to get the most of the measurements this algorithm is a way to fill the frame
before
after
it can already record and choose the save directory
the view also come in 3D, for the usage of point cloud
least but not the last, the depth viewer, currently I havent been using it, since its like the viewer without the color stream control.
Setting of the environment
Pip install pyrealsense2
This is the current easiest way to install it, while the other method is through CMake which I never tested
Other plugins needed will be: numpy, opencv-python, matplotlib
Also can be done with pip install
————————————————————————————————————————
So start scripting:
My first example is to rewrite the Hello World into python
https://github.com/soarwing52/RealsensePython/blob/master/phase%201/Hello%20World.py
Import pyrealsense2 as rs
This is the library import line for realsense, in all further scripts it will be all in this form
The pipeline:
For video processing all the instructions are in a pipeline, so start it and combine the next moves.
Pipeline = rs.pipeline()
pipeline.start()
And then the configurations of the pipeline:
config = rs.config()
config.enable_stream()
This sets up streams of frames which not only when using camera but also when reading from bag files, mainly I use for alignment of frames, other I still haven’t found the function for it
Parameters:
(stream type,resolution, image format, fps)
Example:
config.enable_stream(rs.stream.depth, 640, 360, rs.format.z16, 30)
config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)
The possible can be found in the Intel Realsense Viewer, color,depth,infrared.
One thing I found interesting is that when it in default is the bgr format
in Intel Viewer it shows bgr in rgb, which means where red is blue and the colors are not correct to human eye.
So when I record as rgb8/rgba8, and view in opencv-python, it shows also again wrong, shows rgb in bgr format.
and when I use matplotlib, it showed rgb as default view.
so I need to add one cv2.convert if I want to show RGB image correctly in opencv window.
It is not a big problem just one interesting phenomenon
https://www.learnopencv.com/why-does-opencv-use-bgr-color-format/
RGB/Y16/BGR
In order to get the frames one after one, perform as a stream/video, a try and a while loop is required
Try:
while True:
Except Runtimeerror:
No frames came or ended
Finally:
pipeline.stop()
So for getting the frames
frame = pipeline.wait_for_frames()
depth_frame = frame.get_depth_frame()
color_frame = frame.get_color_frame()
This is quite straightforward
The get frame options for pipeline are:
poll_for_frames()
Get frames immediately
Somehow I can’t make it work, it just always reply no frames coming until the end
wait_for_frames()
It will block the stream until the next stream came, this caused a matching problem which I will describe later when I tested measuring
try_wait_for_frames
I haven’t really get to know this one yet
This is also same options for the syncer, but as I wrote before, I can’t get the syncer to function yet.
For frame queue there are more options
rs.frame_queue()
This will put the frames first in memory and then process and save, if there’s a lot of frames while streaming, this is the method to prevent drop frames since it saves the frame data first.
In this project I didn’t need this function, just came across it while searching
So back to the Hello World sample.
width = depth_frame.get_width()
height = depth_frame.get_height()
dist = depth_frame.get_distance(width / 2, height / 2)
This is getting attribute from frames
The options are
get_width
get_height
get_stride_in_bytes
get_bits_per_pixel
get_bytes_per_pixel
And for the metadata of the frames:
var = rs.frame.get_frame_number(color_frame)
print ‘frame number: ‘+ str(var)
time_stamp = rs.frame.get_timestamp(color_frame)
time = datetime.now()
print ‘timestamp: ‘ + str(time_stamp)
domain = rs.frame.get_frame_timestamp_domain(color_frame)
print domain
meta = rs.frame.get_data(color_frame)
print ‘metadata: ‘ + str(meta)
Frame number, timestamp are the ones I used, others I haven’t get my hands on yet
The get_distance is only for depth_frames while the others apply to all other frames.
So this can get the basic distance. But for accuracy, even though it shows the number up to 76,
The detecting distance of D435 is 10 meters, and for my test, it’s accurate within 5-6m up to +-10cm on measuring objects
key = cv2.waitKey(1)
This is window controlling, with 1 means 1ms the window will show, and 0 means forever
if key & 0xFF == ord(‘q’) or key == 27:
Break
This means when esc/q is pressed, exit the while loop, and proceed to next frame because there is a try outside of the while loop.
Jaythecheyi 