Developing an Archer Search Engine - Part 2
Hello, I have made great progress on the archer search engine. Including, near complete understanding of the scoring system, a working demo, and more! This post is gonna highlight work from February 25, 2026 to March 6, 2026 and it is going to be in chronological order (unlike the previous post).
Day 1
My first goal for this post was to finish implementing the scoring system. I started by setting up logging for every unknown value. My process effectively went like this: Find issue from logs, compare with between ends site, guesstimate what the correct answer was. Here are the discoveries I made:
- E: empty (so obvious)
- W: Equal to an X (I think it's a 3d archery thing)
- Z: Equal to an X (No idea why)
- b: Was a 12 (I think its a 3d archery thing)
- Should it count as an 10 or an X in the global stats? I'm not really sure.
- When I was figuring this out, I kept getting the running total confused with the arrow values, because the data formatting was confusing. I ended up just cutting the data into parts and comparing the individual parts to figure out how this one works.
- m: Same as empty
- For some reason I could only find this in official USA archery tournaments. That might have been a coincidence, but I have no idea as to why. It doesn't appear as anything on the scorecard, similar to E, but unlike M. My best guess is it's the same as empty.
- a: 11
- Apparently some places count inner tens as 11's and I think that's where this is from.
- d
- After seeing the first "d" value, I took a look at some of the data, and some of the archers had 43/40 arrows. I had no idea why this could be the case, especially since, the scorecard didn't display the other values. The last 3 values also didn't seem to affect the score much, if at all. I moved to the data side, and I ended up seeing a bunch of archers randomly having extra values (up to 5 or 6 sometimes). I couldn't think of a possible reason you would have 5 extra values, and why it occurred only in 3d tournaments (yes it was another thing with 3d thing, of course). What was interesting though, was that "d" only EVER appeared in those extra values. That was just very confusing to me. I decided to ignore it and move on. Example (Archer: Aric Clements).
- !
- Also appeared in the same context as "d", where it would only be in the extra data.
Day 2
I began day two with the mission of figuring out the other formats of the score data. Remember how I talked about, in the last post, there being only a few event types I could find. Turns out there is another type, and the score parsing is a little different. These are known as special events. (Example). Originally I tried loading these with /event/ instead of /specialevent/ in the URL, and they didn't load, so I thought they were broken, but after more investigating turns out I'm just dumb.
Once I took a deeper look into special events I was really confused. The score values could go from being in the tens to the thousands. I still haven't figured out the root cause for this, if you have any idea go ahead and email me (located at the bottom of the post).
After that, I was finally at the last step of the score parsing. In this case, the data was very similar to the primary type, but instead it also had a "tb" and an "sts" value. Sts seemed to be a boolean(as in true or false). I came to this conclusion, because the only two present values were 1 and 0, which are sometimes used to represent booleans. Tb was a mystery to me. It was always two characters, and usually "EE" (as in empty value, empty value, most likely). One time I caught a case of it being "6E". I believe it is some type of makeup or bonus end, because it is usually empty. The last idea I had was "tb" could stand for tiebreaker, but I'm still not sure. Looking past the data, the site, for cases of this, always were the match event type, and they included quarter finals, semi finals, etc (Example). Since there were a bunch of categories with different scores for each person, I checked whether there were multiple occurrences of the same archer identifier, but there wasn't. This meant there were multiple occurrences of the same name, which were linked to different identifiers. I investigated more, and I found the scores for the qualifier round located in the event info, instead of the scores(no idea why). It was named "qrnk", which I assume means qualifier rankings. Along with the whole qualifier ranks thing, I found the aforementioned multiple name occurrences, as well. When I looked back at the website I saw something that really confused me. Sometimes there were scores with a number in parenthesis, and the total score was less than ten. Alongside these occurrences, there were totally normal archery scores. After comparing the data and the site, I realized how it works. Each person is grouped into a pair and per end, whoever in one pair has higher arrow values gets points. (2 for having more, 1 each for a tie, and 0 for loosing). That's why all the scores were less than 10! Although this looked confusing on the site, the data was formatted about the same, so it wasn't an issue for me, when it comes to parsing. All that research led me to completion of the rest of the score script(except for the bits which I am yet to understand, mentioned earlier). By the way, I decided to include "tb" as part of the score, because it empty and in turn meaningless most of the time, so it probably counted when it wasn't empty.
Day 3
On day three, I ended up discovering the holy grail of this project. The scoring calculator code (turns out it wasn't as confusing or complicated as I thought).
Along with these two files there was a bit of extra code, but it was easily replicable from just an understanding of archery. I wanna highlight how much I was able to discover from these files.
- SRL = Scoring Rules
- STB = Special Tie Breaker
- NOS = Number of Sessions
- DLD = Download
- ENM = Event name
- RDS = Number of Rounds
- APE = Arrows per end.
This is just scratching the surface. There were countless other things that were explained from this. All that confusion from the earlier days, could be resolved with this code. Although, I haven't put in the time to figure out what my previous confusions in this post truly meant.
Now I was at a point, where I reconsidered what to do with this project. I originally planned to recreate the whole score calculator, but now that it was sitting at my fingertips, there were so many other ways I could make this search engine. One option was giving up all the work I had done before, and just taking the score data by using a headless browser on the site, although that is super expensive and inefficient, for something as simple as this. I concluded the best course of action, after about a week of on and off deliberating, would be to use the aforementioned scoring files, but rewrite the front end score viewer bit. (What I mentioned as "the extra bit of code".) I also decided, for now, I was gonna not store any duplicate data. The only part I would store, would be an index of full name -> List of participated events. From there, I would fetch the events from between ends and calculate the scores manually.
Day 4
This was the day I started working on version one of the site. I produced the full name to event file, using my scripts, and I uploaded it here. Here are some photos of the site:
As you can see, there are a number of issues to note with version one of this project. First of all, the site is really ugly. Next, it barely displays any information, just the average arrow value. Next, it only shows the event names not the tournament names. Next, it doesn't support the aforementioned, special events. I also would like to note that hovering on the event name will show the archer's average arrow value for just that event. Also, there is an issue where random events you didn't participate in are present, but I band-aid solved this by hiding events you only got misses in. I believe the root cause of this issue is that the code for getting the full name -> event file, was based off of the full name -> tournament file, so it includes events you didn't even participate in. A rewrite would definitely solve this. One of the even bigger issues is the speed. For some people it only takes around ten seconds to search, but for some people it can take over a minute. It really depends, and this is solely caused by the way the project is made. Not to mention, the data isn't even real time, due to the full name to event list being pre-calculated. The solution to this and many other problems is a goal of this project I like to call "realtime". This is a non-existant feature which is planned (hopefully) for the next blog post. It would entail cloning the entire between ends database on my own servers, and updating it in real time. This would genuinely make the project perfect, and it's my golden standard I want to achieve, before showing off this project to others. (Also you might have noticed the project is named beagle, this is combination of "Between Ends" + "Archery" + "Google", also they're a goated dog breed.)
Afterword
One thing that is also on my radar to fix is the dependence on names. If two people have the same name, they are treated as the same person. This definitely could be solved using the "altId" value provided by Between Ends, which isn't tournament dependent; however, this value isn't present in every tournament, so I don't really know how to cope with that. I would also like to showcase with the new global statistics:
- Total Arrows Show: 25,473,603
- Total Xs: 842,651
- Total 10s: 4,697,711
- Total 9s: 6,388,899
- Total score: 172,922,845
- Average arrow value: 6.788398820427154
For those interested, I have decided not to release version one, because it was so sketchy, but during the next post I plan to buy a domain and release it, finally. I will also open source all the code, alongside the release.
My email for notes / questions: archeryguy@tuta.com