Skip to content
PreciSim
quickstart

Your first experiment (30 minutes)

With no hardware attached, run a pixel-size calibration chain and see how far your result is from the truth.

This page has exactly one goal: to put an error figure in front of you within 30 minutes. Not "the software starts", but "my pixel size is 0.0015 µm/px away from the truth".

Why pixel size first

Pixel size (µm/px) is the most basic chain in 2D vision: it converts pixel distances in the image into millimetres on the stage. Everything after it — positioning, alignment, measurement — rests on it. It is also the easiest one to get wrong, and getting it wrong raises no error; it just shows up on the line as "occasionally a bit off".

Steps

1. Confirm the virtual machine

On start-up the software assembles devices from the Device section of appsettings.json, which defaults to the virtual machine demo-xy-galvo. The status bar shows the machine name and "connected".

X / Y / Z    travel ±165 / ±120 / 80-140 mm, repeatability 0.2 µm, read noise 0.5 µm
Camera       4096 x 3000, pixel size 1.7236 µm/px (truth), mounting angle -0.80 deg (truth)
Target       dot grid, 1.0 mm dots on a 2.0 mm pitch

Those two truth values are what this chain has to measure. Do not open the machine file yet — you will use them to check your answer in a moment.

2. Start the wizard

Menu Calibration → Pixel size. Three steps:

Step What happens What to watch
Acquire the stage walks a grid of N positions, grabbing a frame and finding circle centres at each progress, centres found per point; cancel mid-run and the stage returns to the start
Solve least squares for the 2x2 Jacobian, decomposed into pixel size X/Y, two direction angles and non-orthogonality the residual
Apply the result is written to the store (SQLite, append only) the record confirmation

Leave the defaults, press Acquire, then Solve.

3. Read the result

pixel size X  1.7251 µm/px          theta_x  -0.798 deg
pixel size Y  1.7248 µm/px          theta_y  -0.801 deg
non-orthogonality 0.003 deg         residual 0.31 µm
truth compare  pixel size 1.7236 µm/px   error 0.0015   tolerance 0.002  -> PASS

The last line is the point. The virtual machine knows the truth, so it can tell you directly whether you calibrated accurately. On real hardware that line does not exist — which is exactly why you practise here first.

4. Break it on purpose

Do not skip this step; it is worth more than the previous three. Shrink the sampling grid (set the point spacing much smaller) and run Acquire and Solve again.

You will see:

  • the residual barely changes, possibly improves;
  • the truth-compare error grows clearly, possibly to a FAIL.

That is the most direct demonstration of residual is not error: the model explains the data you collected perfectly well, but the data spans too little to support an accurate solve. Beginners almost always do the opposite — a dense cluster in the centre of the field, followed by surprise that it is not accurate.

5. Look at the mounting angle

Change the camera's rotationDeg in the machine file from −0.80 to −5.0, restart, and run again. theta_x / theta_y should follow to about −5°. If they come out at +5°, a sign is flipped — which is precisely why the coordinate conventions live in the comments and are held in place by truth-based tests.

Next

Last updated: Sep 22, 2026
Was this page helpful?