<p>If you can believe it: a few neat tools I made to <em>get better items
in the game Diablo 2</em> are the bedrock of my programming career.</p>
<p>This post is about the Optical Character Recognition (OCR) tool I made to
pickup "good items" in D2 and encouraging kids to pursue weird passions.</p>

<h1>Getting started</h1>
<p>When I was a freshman in highschool one of my big passions was the game Diablo 2.</p>
<p>D2 a game where as a hero you collect items and defeat horrifying monsters, moving from
stage to stage until the game is "finished." What makes Diablo 2 so fun, is the
replayability -- it is so badass to come back to a difficult monster and blow it up
in a couple hits.</p>
<p>The problem is it takes a long time to get good items, maybe a thousand different
"runs" where you beat a boss and until you find what you're looking for. I set out
to make tools that automated this process and make it easier.</p>
<blockquote>
<p><em>"Oh it'd be so cool if I could make a program that automatically picked up good items
in this game I am playing..!"</em></p>
</blockquote>
<h1>Items</h1>
<p>Items in Diablo 2 have a few features. A basic item looks something like this:</p>
<pre><code>[&#x3C;prefix>] &#x3C;name> [&#x3C;suffix>]
&#x3C;type>
&#x3C;level>
&#x3C;attributes>
</code></pre>
<p><em>(where [] around something means it is optional)</em></p>
<p>Here's a "Rare heavy belt" (you can tell it's rare because of the yellow item name)</p>
<div style="text-align: center;">
    <img src="/assets/blog/2021-01-diablo-2/stone_buckle_belt.jpg" class="img-bordered">
</div>
<p>Here's a "magical cap" (because of the blue colored name)</p>
<div style="text-align: center;">
    <img src="/assets/blog/2021-01-diablo-2/beryl_cap_of_energy.jpg" class="img-bordered">
</div>
<p>On the magical item, you can clearly see how the item received it's properties. <strong>Beryl</strong>
gives an item "Poison Resistance +5%" and <strong>of Energy</strong> gives "+1 to energy." Neat!</p>
<h1>Pickup the item, or not?!</h1>
<p>This is the big question! In each "run" of Diablo 2 you may kill a few champion sets of
monsters or many bosses, potentially seeing hundreds or thousands of items laying on
the ground.</p>
<div style="text-align: center;">
    <img src="/assets/blog/2021-01-diablo-2/items_on_ground.jpg" class="img-bordered">
</div>
<p>Further compounding the problem of many items on the ground, is most items that first
seem appealing do not have the actual attributes you want. When an item is on the ground
you can only see the name, not things like how much strength the item can give you.</p>
<p>It becomes very tedious to have to pickup each of these items, identify them, make sure
it's good enough, and keep it around on your character long enough to make it useful. You
may find the perfect armor with the number of "socket holes" to put your "runes" into,
but you need 5,000 more runs to find the right runes.</p>
<p>If only you had a tool to help pickup "worthy items" then that would make it so much faster
to get through each run. Or... completely automate the runs!</p>
<h3>Determining worthiness</h3>
<ol>
<li>Does the item name or quality match a filter? Is it a <em>rare</em> helm or a <em>magic</em> one?</li>
<li>After picking up the item, does it have all the right attributes?</li>
<li>Now where do we put it?!</li>
</ol>
<p>In another post I may go into how I did my pathfinding and solved other problems,
I'll <em>try</em> to focus on the OCR problem #1 and #2 for now! #3 is a pretty fun
problem as well, organizing the inventory and finding space for items.</p>
<h1>How my OCR worked</h1>
<p>Because my first attempts to automate Diablo 2 could only see pixels, not in game memory,
I had to make a tool to turn the pixels into text -- AKA an OCR.</p>
<h3>Get lines</h3>
<p>Scan horizontally splitting up each line</p>
<div style="text-align: center;">
    <img src="/assets/blog/2021-01-diablo-2/items_on_ground_2.jpg" class="img-bordered">
</div>
<h3>Get characters</h3>
<p>They are 2px apart exactly (thankfully!); if > 2px apart it's a new item</p>
<div style="text-align: center;">
    <img src="/assets/blog/2021-01-diablo-2/get_characters.jpg" class="img-bordered">
</div>
<h3>"Hash" each character to a string</h3>
<p>I can "define" characters by the distance of each pixel from the top left.</p>
<p>For example, if we get these pixels back, where "@" is part of the character we're
identifying:</p>
<pre><code>@ . @
@ . @
@ . @    
@ @ @   ↑
. . .   y

x →
</code></pre>
<p>The string representing these @ coordinates would look like...</p>
<pre><code>(x, y)
1,1; 3,1;
1,2; 3,2;
1,3; 3,3;
1,4; 2,4; 3,4;
</code></pre>
<p>Or smashed into a shorter string, spaces removed, and set to a variable..</p>
<pre><code class="language-python">u = "1,1;3,1;1,2;3,2;1,3;3,3;1,4;2,4;3,4;"
</code></pre>
<h3>Compare that hash to known list of characters</h3>
<p>I made a small program that automated the process of "hashing" each letter. I made ABC...
in Diablo 2's font and turned each character into this "hash string" of coordinates.</p>
<p>Each hash string returned a character, so <code>"1,1;3,1;1,2;3,2;1,3;3,3;1,4;2,4;3,4;"</code> was turned
into "u" and the same for each character.</p>
<h3>We have an item!</h3>
<p>Each of these characters was returned and combined into a string, representing the item.</p>
<p>We also already know the color of the item, so given the color of the item and its name
we can pass the first part of our "worthiness" check!</p>
<p>After we pickup the item from the ground, we run this same OCR process to get the
item attributes and make sure everything matches what we want. Ta da!</p>
<h1>An aside: encourage your kids</h1>
<h3>A scary obsession or fuel for a life long career?</h3>
<p>Imagine your kid spending 8-16hrs a day on a computer playing Diablo 2 with hundreds
of pieces of scratch paper around them. I would jot down ideas, maps, items, recipes
for the game, programming strategies, etc.</p>
<p>It's like that scene from Alway's Sunny where Charlie goes nuts:</p>
<div style="text-align: center;">
    <img src="/assets/blog/2021-01-diablo-2/its-alway-sunny-charlie.jpg" class="img-bordered">
</div>
<p>I couldn't sleep, I couldn't <em>not</em> figure out these problems. I would spend weeks attacking
something, starting from scratch over and over until it worked. However untalented
I am with math or general intelligence, I have always been able to rely on
my perseverence against difficult problems.</p>
<p>Wouldn't you be concerned about your kid? I am lucky my father recognized I am pretty
introverted and was actually very happy and fulfilled spending a ton of time alone
working on D2 programs.</p>
<h3>Where would I be today?</h3>
<p>Without all of this effort and pay off from Diablo 2, I am not sure I would be where I am
today. I learned PHP, C#, C++, AutoIt, Javascript, etc. because of my enjoyment in
this silly game.</p>
<p>Imagine that kid being denied Diablo 2 or all of that fun, instead being forced into
something I wasn't passionate about. Thank God my father believed in me and was confident
I was on a productive path.</p>
<h3>Carefully encourage your kids</h3>
<p>I am sure if <em>all</em> I did was play D2 and never learn any programming, that would be a
problem. It's important to make your kids "play" into learning, don't just let them
play Fortnite all day. Maybe they love some aspect of Fortnite that you can turn
into a learning opportunity.</p>
<p>Leverage your kids' passion for their future! Whether that's Dungeons and Dragons, football,
or <a href="https://www.ted.com/talks/sir_ken_robinson_do_schools_kill_creativity?language=en">dancing.</a></p>

Diablo 2 OCR