MLX90640 IR Thermal Camera Checkerboard artifact

General project help for Adafruit customers

Moderators: adafruit_support_bill, adafruit

Please be positive and constructive with your questions and comments.
Locked
User avatar
aklianos
 
Posts: 1
Joined: Thu Jul 21, 2022 1:30 pm

MLX90640 IR Thermal Camera Checkerboard artifact

Post by aklianos »

Howdy all,

We have been playing around with the MLX90640 IR sensor and there is a weird
checkerboard artifact going on.
(I am not talking about the low resolution of the sensor).
but the hot-cold pixels that appear in a checkerboard pattern (e.g. bottom right corner).
Any idea if this is coming from the sensor array or from the code after the data capture?
Attachments
Checkboad artifact issue- MLX90640.jpg
Checkboad artifact issue- MLX90640.jpg (77.56 KiB) Viewed 652 times

User avatar
adafruit_support_mike
 
Posts: 67446
Joined: Thu Feb 11, 2010 2:51 pm

Re: MLX90640 IR Thermal Camera Checkerboard artifact

Post by adafruit_support_mike »

That's almost certainly an artifact of the code generating the image.

The text at the top of the image says it's using 'inter cubic' interpolation, which is most likely to be an uncommon name for bicubic interpolation.. a process that generates a smooth curve that passes through four points, calculated vertically and horizontally for each point in the original image.

Interpolation schemes have trouble generating good values for points that aren't directly in line with a value from the original image, and the image above looks like the algorithm is really struggling.

Try scaling the image up in multiple steps, increasing the resolution by a factor of 3 each time. Hopefully interpolating across a smaller distance will generatre enough good values to avoid the checkerboard effect.

User avatar
Disciple
 
Posts: 852
Joined: Tue Jan 06, 2015 8:13 pm

Re: MLX90640 IR Thermal Camera Checkerboard artifact

Post by Disciple »

A properly functioning MLX90640 can still show a checkerboard effect when the thermal scene before it changes. The sensor captures its pixels of thermal data in two passes, half the pixels update followed by the other half slightly later. An internal register can be changed to cause the two groups of pixels to update in either a checkerboard pattern or a horizontal stripes pattern. Either way, warm bodies in motion will cause a pattern to appear. Scaling up the pixels with higher order interpolation, they'll soften the sharp edges of the pattern to a degree, but it's a real part of the raw data.
Eliminating the pattern would take a bit of fancy work, like separating the pixels into two half images, then generating the missing pixels by averaging their neighbors to produce two full images to display... a fair task but possible. Hope this helps.

Hallelujah!
Disciple

User avatar
sj_remington
 
Posts: 997
Joined: Mon Jul 27, 2020 4:51 pm

Re: MLX90640 IR Thermal Camera Checkerboard artifact

Post by sj_remington »

By default, the sensor read out is interleaved checkerboard, in two passes, but I've rarely seen the effect shown in the image. As mentioned above, it can appear when the "hot spot" source is moving.

If the checkerboard pattern is consistent, that could be due to a defective sensor, or faulty code. Please post the raw sensor readings, in code tags, and the code as well.

User avatar
Sneha_Papineni
 
Posts: 9
Joined: Tue Jul 26, 2022 3:03 pm

Re: MLX90640 IR Thermal Camera Checkerboard artifact

Post by Sneha_Papineni »

Howdy all,

PFA for the requested files. Also, the captured image below doesn't have any interpolation technique , which can be seen in the code (yet, the check board artifact is still visible)

sensor capture :
Sensor capture
Sensor capture
pic_2022-07-21_17-20-42.972210.jpg (67.46 KiB) Viewed 552 times
Code :
mlx90640 modified code.py
code
(13.53 KiB) Downloaded 11 times
Raw temperature readings (which are temperature readings from right to left in the capture) :
raw temperature data.py
Raw Temperature data
(4.53 KiB) Downloaded 12 times
Also, can you please provide the emissivity information that is set in the sensor and Standard Procedure for calibrating the thermal sensor.

Thanks !

User avatar
sj_remington
 
Posts: 997
Joined: Mon Jul 27, 2020 4:51 pm

Re: MLX90640 IR Thermal Camera Checkerboard artifact

Post by sj_remington »

The raw data definitely show a checkerboard pattern. You might try the Arduino C/C++ example code to see if there is a problem with python.

Raw data were processed by Matlab to produce a simple grayscale image.

Code: Select all

data = csvread("mlx.csv");
datamin = min(data(1:24,1:32),[],'all');
datamax = max(data(1:24,1:32),[],'all');
I = mat2gray(data(1:24,1:32),[datamin datamax]);
imshow(I);
imwrite(I, 'image.png');
grayscale.PNG
grayscale.PNG (25.91 KiB) Viewed 547 times

User avatar
Sneha_Papineni
 
Posts: 9
Joined: Tue Jul 26, 2022 3:03 pm

Re: MLX90640 IR Thermal Camera Checkerboard artifact

Post by Sneha_Papineni »

Can you please share how to obtain the raw data of the sensor (i.e., not the apparent temperature value that is processed /calculated and given as output) ?

Thank you

User avatar
adafruit_support_mike
 
Posts: 67446
Joined: Thu Feb 11, 2010 2:51 pm

Re: MLX90640 IR Thermal Camera Checkerboard artifact

Post by adafruit_support_mike »

Also try splitting out the even rows and even columns, assuming the row/column values start at 1.. if the numbers start at 0 get an image that contains the odd rows and columns.

It looks like a lot of those pixels have the same value.

User avatar
Sneha_Papineni
 
Posts: 9
Joined: Tue Jul 26, 2022 3:03 pm

Re: MLX90640 IR Thermal Camera Checkerboard artifact

Post by Sneha_Papineni »

Howdy !

PFA for the images after splitting the even columns and rows (i.e., reduced the resolution to 12*16 from 24*32) and the original image.
Grayscale image obtained after splitting the even rows and columns together.
Grayscale image obtained after splitting the even rows and columns together.
12x16.png (25.34 KiB) Viewed 520 times
mlx (12x16).csv
csv file after splitting
(930 Bytes) Downloaded 2 times
original capture (24 * 32 resolution)
original capture (24 * 32 resolution)
24x32.png (27.54 KiB) Viewed 520 times
The new / less resolution image is better in terms of minimal to no checkerboard artifacts. Yet, it has very poor resolution due to which it is difficult to get true temperature values at a specified target. How could we solve this problem and get a decent capture with 24*32 pixels resolution and without this check-board artifacts.

Thank you and looking forward to hear from you

User avatar
adafruit_support_mike
 
Posts: 67446
Joined: Thu Feb 11, 2010 2:51 pm

Re: MLX90640 IR Thermal Camera Checkerboard artifact

Post by adafruit_support_mike »

Try extracting the other complementary images:

- even rows, odd columns
- odd rows, even columns
- odd rows, odd columns

The large image looks like one of those groups has much lower variation than the others.

User avatar
Sneha_Papineni
 
Posts: 9
Joined: Tue Jul 26, 2022 3:03 pm

Re: MLX90640 IR Thermal Camera Checkerboard artifact

Post by Sneha_Papineni »

Howdy!
The maximum frame rate that I could get for capturing thermal images is 2 Frames per second. Yet, to get the raw temperature data using the attached code, the frame rate dropped to 1 frame per second. Why is that so ?

Can you please share if there is any other code/ way to get the the raw temperature data at a faster frame rate?


import time
import board
import busio
import adafruit_mlx90640

i2c = busio.I2C(board.SCL, board.SDA, frequency=800000)

mlx = adafruit_mlx90640.MLX90640(i2c)
print("MLX addr detected on I2C", [hex(i) for i in mlx.serial_number])

mlx.refresh_rate = adafruit_mlx90640.RefreshRate.REFRESH_2_HZ

frame = [0] * 768
while True:
try:
mlx.getFrame(frame)
except ValueError:
continue

for h in range(24):
for w in range(32):
t = frame[h*32 + w]
print("%0.1f, " % t, end="")
print()
print()

User avatar
Sneha_Papineni
 
Posts: 9
Joined: Tue Jul 26, 2022 3:03 pm

Re: MLX90640 IR Thermal Camera Checkerboard artifact

Post by Sneha_Papineni »

Howdy!

I am writing this regarding the faulty mlx90640 thermal sensor we received. PFA for a thermal capture of a close up shot of hot water (inside a cup/mug) which is around (76 degrees Celsius - measured using a K-type thermocouple at the centre of the frame).
22.py
Raw temp. values of the capture
(4.52 KiB) Not downloaded yet
Thermal capture
Thermal capture
pic_2022-08-15_20-40-22.620782.jpg (68.65 KiB) Viewed 386 times
Also, find the raw temperature values of the capture. From the values you can see the extreme variations of the temperature values of each pixel (whereas, ideally they should be more or less ,the same). Such huge deviations (Max temp. pixel - min temp. ~ 10 degrees difference) despite the error - max.temp. - thermocouple temp (~3 degrees celsius), Such high discrepancies are not accepted for the research purpose. We need another sensor in compensation for the faulty one. We will return the current sensor once you sent the new sensor.
Adafruit.pptx
Challenges
(451.82 KiB) Downloaded 5 times
It has almost been two months since we started bothering you regarding these faulty results/checkerboard patterns. Moreover, we haven't yet received any satisfactory reply and thus is not expected from your company. Kindly treat this on high priority and send us a new non-faulty thermal sensor.

Sincerely,
Sneha Papineni

User avatar
adafruit_support_mike
 
Posts: 67446
Joined: Thu Feb 11, 2010 2:51 pm

Re: MLX90640 IR Thermal Camera Checkerboard artifact

Post by adafruit_support_mike »

I haven't seen even similar behavior from any MLX90640, and the information from the debugging is inconclusive.

We'll do a replacement, but if the second one shows the same kind of behavior we'll have to assume it's related to conditions in the operating environment. Send a note containing a link to this page and your order number to [email protected]. The folks there will get you a replacement.

User avatar
Sneha_Papineni
 
Posts: 9
Joined: Tue Jul 26, 2022 3:03 pm

Re: MLX90640 IR Thermal Camera Checkerboard artifact

Post by Sneha_Papineni »

Howdy!

Thank you very much for your support and understanding. I have mailed the requested details to [email protected]

Please let me know if anything else is required from for the timely replacement.

Thanks and regards,
Sneha Papineni

Locked
Please be positive and constructive with your questions and comments.

Return to “General Project help”