Diablo 2 OCR

If you can believe it: a few neat tools I made to get better items in the game Diablo 2 are the bedrock of my programming career.

This post is about the Optical Character Recognition (OCR) tool I made to pickup “good items” in D2 and encouraging kids to pursue weird passions.

Getting started

When I was a freshman in highschool one of my big passions was the game Diablo 2.

D2 a game where as a hero you collect items and defeat horrifying monsters, moving from stage to stage until the game is “finished.” What makes Diablo 2 so fun, is the replayability – it is so badass to come back to a difficult monster and blow it up in a couple hits.

The problem is it takes a long time to get good items, maybe a thousand different “runs” where you beat a boss and until you find what you’re looking for. I set out to make tools that automated this process and make it easier.

“Oh it’d be so cool if I could make a program that automatically picked up good items in this game I am playing..!”


Items in Diablo 2 have a few features. A basic item looks something like this:

[<prefix>] <name> [<suffix>]

(where [] around something means it is optional)

Here’s a “Rare heavy belt” (you can tell it’s rare because of the yellow item name)

Here’s a “magical cap” (because of the blue colored name)

On the magical item, you can clearly see how the item received it’s properties. Beryl gives an item “Poison Resistance +5%” and of Energy gives “+1 to energy.” Neat!

Pickup the item, or not?!

This is the big question! In each “run” of Diablo 2 you may kill a few champion sets of monsters or many bosses, potentially seeing hundreds or thousands of items laying on the ground.

Further compounding the problem of many items on the ground, is most items that first seem appealing do not have the actual attributes you want. When an item is on the ground you can only see the name, not things like how much strength the item can give you.

It becomes very tedious to have to pickup each of these items, identify them, make sure it’s good enough, and keep it around on your character long enough to make it useful. You may find the perfect armor with the number of “socket holes” to put your “runes” into, but you need 5,000 more runs to find the right runes.

If only you had a tool to help pickup “worthy items” then that would make it so much faster to get through each run. Or… completely automate the runs!

Determining worthiness

  1. Does the item name or quality match a filter? Is it a rare helm or a magic one?
  2. After picking up the item, does it have all the right attributes?
  3. Now where do we put it?!

In another post I may go into how I did my pathfinding and solved other problems, I’ll try to focus on the OCR problem #1 and #2 for now! #3 is a pretty fun problem as well, organizing the inventory and finding space for items.

How my OCR worked

Because my first attempts to automate Diablo 2 could only see pixels, not in game memory, I had to make a tool to turn the pixels into text – AKA an OCR.

Get lines

Scan horizontally splitting up each line

Get characters

They are 2px apart exactly (thankfully!); if > 2px apart it’s a new item

“Hash” each character to a string

I can “define” characters by the distance of each pixel from the top left.

For example, if we get these pixels back, where “@” is part of the character we’re identifying:

@ . @
@ . @
@ . @    
@ @ @   ↑
. . .   y

x →

The string representing these @ coordinates would look like…

(x, y)
1,1; 3,1;
1,2; 3,2;
1,3; 3,3;
1,4; 2,4; 3,4;

Or smashed into a shorter string, spaces removed, and set to a variable..

u = "1,1;3,1;1,2;3,2;1,3;3,3;1,4;2,4;3,4;"

Compare that hash to known list of characters

I made a small program that automated the process of “hashing” each letter. I made ABC… in Diablo 2’s font and turned each character into this “hash string” of coordinates.

Each hash string returned a character, so "1,1;3,1;1,2;3,2;1,3;3,3;1,4;2,4;3,4;" was turned into “u” and the same for each character.

We have an item!

Each of these characters was returned and combined into a string, representing the item.

We also already know the color of the item, so given the color of the item and its name we can pass the first part of our “worthiness” check!

After we pickup the item from the ground, we run this same OCR process to get the item attributes and make sure everything matches what we want. Ta da!

An aside: encourage your kids

A scary obsession or fuel for a life long career?

Imagine your kid spending 8-16hrs a day on a computer playing Diablo 2 with hundreds of pieces of scratch paper around them. I would jot down ideas, maps, items, recipes for the game, programming strategies, etc.

It’s like that scene from Alway’s Sunny where Charlie goes nuts:

I couldn’t sleep, I couldn’t not figure out these problems. I would spend weeks attacking something, starting from scratch over and over until it worked. However untalented I am with math or general intelligence, I have always been able to rely on my perseverence against difficult problems.

Wouldn’t you be concerned about your kid? I am lucky my father recognized I am pretty introverted and was actually very happy and fulfilled spending a ton of time alone working on D2 programs.

Where would I be today?

Without all of this effort and pay off from Diablo 2, I am not sure I would be where I am today. I learned PHP, C#, C++, AutoIt, Javascript, etc. because of my enjoyment in this silly game.

Imagine that kid being denied Diablo 2 or all of that fun, instead being forced into something I wasn’t passionate about. Thank God my father believed in me and was confident I was on a productive path.

Carefully encourage your kids

I am sure if all I did was play D2 and never learn any programming, that would be a problem. It’s important to make your kids “play” into learning, don’t just let them play Fortnite all day. Maybe they love some aspect of Fortnite that you can turn into a learning opportunity.

Leverage your kids’ passion for their future! Whether that’s Dungeons and Dragons, football, or dancing.