Vertical Coordinate Offset During Drawing in OSD/Annotorious Integration

[itayG98]

I am developing react 19 MUI 7 app with OSD 5.0.1. and annotorious 3.8.6

I am experiencing a consistent vertical coordinate offset when drawing new annotations.
this issue is strictly limited to the "drawing" phase; existing annotations remain correctly aligned when dragged or manipulated. Furthermore, the behavior is observed exclusively on wide, non-square images, suggesting an aspect-ratio-related coordinate transformation discrepancy.

my code is similar to this :

export const MyComp = ({ options }) => {
  const viewerRef = useRef(null);
  const osdInstance = useRef(null);
  const annoInstance = useRef(null);

  useEffect(() => {
    if (osdInstance.current) return;

    // Initialize OSD Viewer
    osdInstance.current = new OpenSeadragon.Viewer({
      element: viewerRef.current,
      ...options,
    });

    // Initialize Annotorious
    annoInstance.current = Annotorious(osdInstance.current, {
});

    // Cleanup
    return () => {
      if (annoInstance.current) {
        annoInstance.current.destroy();
        annoInstance.current = null;
      }
      if (osdInstance.current) {
        osdInstance.current.destroy();
        osdInstance.current = null;
      }
    };
  }, [options]);

  return (
    <Box
      ref={viewerRef}
      sx={{
        width: '100%',
        height: '100%', 
      }}
    />
  );
};

[rsimon]

Sounds like it could be a CSS-related issue. Can you confirm that the SVG element that Annotorious positions over the OSD viewer is properly aligned, or if CSS from outside might intervene with the placement? If not, can you provide an online example that reproduces the issue?

[itayG98]

I will try to create a demo soon

Any way the element of the viewer match the annotations layer by width height and placement

[itayG98]

I've created a demo on CodeSandbox to isolate an offset issue I'm having: Demo

The Bug:
It happens only within my own web app
There is a vertical offset when drawing or selecting shapes on non-squared images.

Observed Behavior:

While drawing a polygon, every click registers with a vertical offset from between my navigator and the point .

After finishing the shape and deselecting it, the polygon resets to the correct position (the exact points I clicked).

It seems the element holding the active polygon points has the wrong placement/transform applied during the editable state. Once deselected, it drops down to the base canvas and aligns perfectly.

I hope may be a tip/hint from you can help me with this struggle

thank you anyway!

[rsimon]

I'm not sure I'm missing something. But your web demo works as it should. (Just tried with Chrome/Mac - not sure if that's a browser issue.) If the CodeSandbox demo works for you, too, then the question is: where does your app differ from the CodeSandbox demo, in terms of surrounding markup, CSS, etc.

Without seeing an example that actually reproduces your error behavior, there's not much I can suggest, to be honest.

[itayG98]

I managed to reproduce the weird scenario: when I add tiled images dynamically using code, there is an offset if none of the images have 0,0 x and y values.

link : Link

imageUrls.forEach((url, index) => {
  viewer.addTiledImage({
    tileSource: {
      type: "image",
      url: url,
    },
    // This line is problematic
    x: 1 + index * 2, // Offsetting images so they don't overlap completely
    y: 0,
    width: 1,
  });
});

[itayG98]

How to reproduce :

Adding tile image so none of them is at 0,0
click over the image in order to create a shape
the offset will determined by the offset of one of the images in scale of 0-1 where 1 = the tiled image's width

In this example i clicked over the image to cover it
[Demo](https://codesandbox.io/p/sandbox/osd-react-19-annotorious-at-floating-window-forked-4q4dk9

Create/Selected state have offset :

Non-selected state does not have offset

[rsimon]

Ah – yes, that explains it: I'm afraid Annotorious does not currently support multi-image scenarios. The error message in the console hints at this ("[Viewport.viewportToImageCoordinates] is not accurate with multi-image; use TiledImage.viewportToImageCoordinates instead."), but it's fully handled as an actual error case.

I haven't tested myself yet if a single image, placed at a custom world position (i.e. not at x=0, y=0, w=1) also shows this behavior, but it looks like it, given your example above.

Support for multi-image may be on the roadmap at some point, but not currently, I'm afraid.

[itayG98]

I've decided to save a 'Delta' of the first tile for now and subtract it from all the rest i hope it will be a small patch solution for now.
This is a side effect that needs to be checked

[rsimon]

I must admit I haven't looked into the details of this yet. But the proper solution is, likely, not trivial.

Annotorious currently works in image coordinate space (pixels), and assumes that OSD world coordinate space is just a scaled version of the image space. That (as we can see) only holds true if:

• There's only one image
• The image is at the default position (0 / 0, width = 1)

For freely positioned images, Annotorious would have to be re-engineered a bit to work in OSD world space natively; and then derive pixel coordinates in a smart way (meaning it would have to know which image each annotation belongs to).

As I said: may be on the roadmap at some point, but not right now :-(