Classify 松葉ガニ (matsuba, male snow crab) vs ベニズワイガニ (benizuwai, red snow crab) from photos with a neural network, in Go only.
A sibling project of
go-fizzbuzz-tflite: the model
is trained in pure Go with a hand-written backprop/Adam implementation — no
Python and no TensorFlow — and the .tflite flatbuffer is generated directly
from Go, then run with go-tflite.
The two species separate reasonably well by color when raw: matsuba is dull brownish orange, benizuwai is vivid red even before boiling. So instead of a CNN over pixels, the pipeline runs a tiny MLP over color statistics:
featurereduces an image region to 14 numbers: a saturation-weighted hue histogram (12 bins of 15° across the warm half of the hue circle) plus mean saturation and mean value. Only warm-hued, saturated, not-too-dark pixels count as crab; ice, trays, blue coolers and green tarps are excluded by construction. Pixels are center-weighted, since crabs sit near the middle of a frame more often than backgrounds do.cmd/traintrains dense(14→16) tanh, dense(16→3) softmax — 291 parameters — on real photos from Wikimedia Commons (data/, 7 matsuba + 4 benizuwai + 11 negatives), expanding each photo into hundreds of random crops with hue/saturation/value jitter, and writeskani_model.tflite. The third class, その他, is trained on faces, hands, cardboard and wood, so that warm-colored things that are not crabs have somewhere to go other than being forced into a species.main.goproposes regions by hysteresis thresholding — a region must be seeded by vividly saturated warm pixels, then grows through duller ones — and classifies each region with the TensorFlow Lite interpreter. A region is framed only when one species holds a clear plurality (≥45%, tunable with-minprob) of the probability mass; その他 regions and toss-ups are dropped, and if nothing passes the bar the best guess is reported with a?instead of being silently discarded.
Training and inference share the same extraction code, so the model always sees features produced by the exact same code path.
Train (pure Go, no C dependencies). With only 22 photos, accuracy is measured by leave-one-photo-out cross-validation: for every photo a model is trained from scratch on the others and must classify the photo it has never seen.
$ go run ./cmd/train
data: 22 photos (松葉ガニ 7, ベニズワイガニ 4), 320 crops per class
model: dense(14->16) tanh, dense(16->3) softmax, 291 params
cv 1/22 ok data/matsuba/bonnebay.jpg -> 松葉ガニ (84.3%)
...
cv 22/22 NG data/other/wood_texture.jpg -> 松葉ガニ (81.7%)
leave-one-photo-out: 11/22 correct (5m29s)
final training on 954 crops took 159ms
wrote kani_model.tflite
11/22 sounds bleak, but it is the harshest possible reading: every fold holds out an entire photo — a lighting setup, a background, a subject the model has never seen any variation of — and dull matsuba photos, washed-out aquarium benizuwai, gray wood and tanned faces genuinely overlap in color space. The per-region pipeline below, with its plurality rule and その他 filtering, is what actually ships.
Classify photos (requires the TensorFlow Lite C library, see go-tflite):
$ go run . data/benizuwai/teradomari.jpg
data/benizuwai/teradomari.jpg: 1 regions
[1] (0,0)-(960,641): ベニズワイガニ (93.4%)
$ go run . portrait.jpg
portrait.jpg: カニは見つかりませんでした (その他 1 領域)
$ go run . data/matsuba/murakami.jpg
data/matsuba/murakami.jpg: はっきりしません — 松葉ガニ? (44.7%)
A photo with several separated crabs is reported per region
(-whole disables this):
$ go run . mixed.jpg
mixed.jpg: 2 regions
[1] (540,70)-(1500,721): ベニズワイガニ (86.1%)
[2] (0,0)-(480,358): 松葉ガニ (83.1%)
-box additionally writes <image>_box.jpg with every confidently
classified region framed in its species color — green for 松葉ガニ, blue
for ベニズワイガニ — and tagged with its number, confidence and species
name. The Japanese glyphs are 8×8 bitmaps from
美咲フォント, embedded in
fontjp.go, so there is still no font dependency:
$ go run . -box mixed.jpg
...
wrote mixed_box.jpg
This is a color model working from 22 photos, honestly labeled as such:
- It assumes raw crabs. Boiled matsuba turns red as well and will read as benizuwai.
- Tanned skin and matsuba shell genuinely overlap in this feature space. The その他 class and the plurality rule filter most faces (including every face in the photo that motivated them), but a face in crab-like lighting can still sneak through, and dull crab photos pay for the filter by landing in はっきりしません territory.
- One training photo (crabs in a blue cooler on a warm wooden floor) is still confidently misclassified as benizuwai: its floor-lit saturation profile simply is benizuwai-shaped. Strong color casts beat a color model.
- A hand holding the crab is connected to it, so it ends up inside the crab's box; region detection is connectivity-based, and crabs piled on ice merge into one region with one combined answer.
Doing this properly would need a CNN and a real labeled dataset — GPU and Python territory. The point here is how far a 274-parameter pure-Go model gets on real photos.
MIT (code). Training photos are CC-licensed, see data/ATTRIBUTION.md.
Yasuhiro Matsumoto (a.k.a. mattn)