Automation with "visgrep"

"visgrep" is part of the "xautomation" package.

The "visgrep" searches for small images within larger images.
The main difficulty is the lack of clear documention (other than the man page)
and examples.

You can ignore the need for 'pat' image formats, as a 8-bit sRGB PNG image will
work fine with the program.   If it is NOT this exact format you will get
"Read PNG failures"!

Example (using imagemagick "convert" and "mogrify" for image handling)....

    # get a screen shot of the window (see "/info/X/WindowID.txt" )
    id=$( get window id of application )
    convert x:$id -depth 8 png24:screen.png

    # Now use some image editor to get the wanted "object.png" from
    # that screen shot, using a image editor like "xv" or "gimp".
    # Ensure it is 8bit PNG (or "visgrep" will fail wierdly)
    mogrify -depth 8 png24:object.png

    # Check the image is 8 bit and sRGB!
    identify object.png

    # now look for "object" on that (or new) screen shots...
    visgrep screen.png object.png
    # ... list of matches ...


From the coordinates list you can then use something like "xdotool"
to move the mouse to or near image, and click (relative to the window).

    id=$( get window id of application )
    image_cmd="convert x:$id +page png24:-"

    position=$( visgrep <(image_cmd) object.png
    if [[ -z $position ]]; then
      echo >&2 "Object not found"
      exit 1
    fi
    read x y z <<<${position/,/ }  # extract the coords from position info

    # now click, 5 pixels in from the discovered corner of object
    xdotool mousemove -window $id $((x+5)) $((y+5)) sleep 0.1 click 1

    # Doing this with "xte' (the package which provides "visgrep") requires you
    # to calculate using screen coords instead of window coords!

If you can crop the image to search and ass that crop offest to the
resulting position, the search could be made faster still.

An example of this is given in "https://antofthy.gitlab.io/info/apps/xdotool"
for repeated minecract stick making (script "mc_make_sticks") which is still
being slowly improved.  Though the new auto-crafter means this is no longer
needed.

---

You can make the search faster still, by look for a VERY VERY TINY but
distinctive part, in or near the 'object' image, before searching for the
"object" itself,  that is look for a smaller 'detection' image).

"visgrep" will then look for that 'detection' image, then check to see if
there is a match for the larger 'object' at that location.  Which speeds
the search enormously.

    # extract a small but unique component of the 'object'...
    crop object.png -crop 4x4+5+5 +page -depth 8 detect.png

    # Now look for tiny 'detect' image and when found look for 'object'
    visgrep -x-5 -y-5 screen.png detect.png object.png
    # ... list of matches ...

Notes:
  * visgrep is VERY FAST compared to ImageMagick "compare" (more specialised)
  * It uses color vectors to compare images if you want to do 'fuzzy' matching
    though that is not usually needed in games, may be need for text matching
  * Can also look for all matches and multiple objects

WARNING:
  * The PNG file however MUST be '8-bit' PNG. It fails with little to no
    feedback...

"visgrep" actually works best when you have a small library of 'object' images,
and if they all have a common small detection indicator (like a box corner).

It could use some extra options like (stop on first match), and to limit the
area to search in large image, and specify area around the 'detect' image.

------------------------------------------------------------------------------