Miscellanea madness

Hey guys, hope you’re well. This time around I’ll be talking about some of the (relatively) smaller changes that happened to THP in the last few weeks. Yeah, I’m kinda milking the subject a little but I honestly do think that these things are well worth talking about in detail. Kind of the point of behind the scenes access, right?

Archived stories and their themes

This might seem obvious first but every story thread that has been archived is no  longer generated by the board software. Basically meaning that they’re static HTML files stuck in time content-wise. So if any changes are made to the rest of the site, adding new themes, changing scripts and whatever else, these old threads won’t be seeing any of the changes.  

We had a situation for a while that I changed the way themes are processed and so all the old and bloated scripts that were still called up on these old threads stopped working. Fixing that isn’t straightforward. I had to find a pattern that could be matched in every archived thread that would replace these old scripts, the old CSS dropdown selectors and fix a bunch of links while I was at it (some of the earliest threads had replies that pointed to the touhouproject.com domain!). There’s no way that could be done manually due to hundreds if not thousands of files of old threads.  

There was a risk of being too zealous and replacing too much and losing information as well as the risk of replacing too little and not fixing issues. This took way more time than I would have liked to have spent  and writing the new script that sets the theme according to your preferences was the least of my worries. This is why it’s unlikely that any new features for the boards will be backported to the archived stories.  

Coinflips, dice and timers

Conceptually, these aren’t really hard to add into the board software. Whenever a post is processed (see more of that on the article on post previewing) the text string of the message field has a lot done with it, some of it matching and replacing for patterns. So if you know how to do some regular expression you can set a pattern that matches something like 1d6-2 and then points to a function that then does the calculations necessary and returns the output.  

In all honesty, something like the coinflip and the dice functions took like an hour to implement, including all the checks for special situations. What was a bit trickier was getting it to play nice with the rest of the board software. These match and replace functions were written over a decade ago and, in some cases, were using deprecated methods for matching arrays and whatnot. This meant that I had to do a rewrite of some pieces of the mechanism and bring it up to modern standards. Figuring out what I needed to do and when in the code took some effort.  

Propagating these changes to the post preview functionality and other bits of code also required some testing and trial and error. And something like the timer, which is conceptually similar, also required writing user-side javascript to update the time. Optimizing that and making sure that there wouldn’t be errors during common usage took more time to sort out than the actual writing of the basic code.  

Cookies

Much misunderstood, cookies are an important way to store data on user’s computers that interacts with websites. I made two changes relating to them recently: moved the mobile device detection away from a cookie and to local storage and hardened the cookies that were set for remaining functionality.

The former really haas no practical impact other than being one less cookie you have to store and communicate to the site about. May make loading imperceptibly faster (it’s just bytes of data, if that). The latter is also something that won’t affect most people as the remaining cookies are just used for moderation. Still, it was overdue. And I simplified slightly one of the scripts that runs, as it no longer needs a library to check for the mobile cookie.

Importing and exporting settings

I’ll level with you: this is by far the hackiest thing on the site I’ve done. For security purposes, browsers don’t really allow sites to save data onto their filesystem. Otherwise you’d go to one sketchy site and get a million viruses automatically saved to your hard drive. So the solution is creating a temporary fake link that’s clicked when you push the export button. It’s treated as if it were a file you were downloading from somewhere. Of course, for it to be treated as a file you have to create a file. So encoding the localstorage data is fundamental for the thing to work.  

Unfortunately, this kind of thing isn’t that well-documented. I imagine because it has limited use and also because the standard drafted by the W3 took a while to be hashed out properly. Other similar standards came and went and browser adoption was slow. So, yeah, it took me a long time to figure out how to use the functions I needed and how to transform the localstorage data into a human-readable JSON output.  

The import functionality is likewise ad-hoc and weird. I already had something of an inkling on how it was to be done as it shared a lot of the concepts and some functions with the code I was probably going to use for the thumbnails in the post preview (this was before I had done either, in the planning stages). Documentation was poor as well so, while easier as I had already done the export, it still took more time than it should have to figure out how to process the data appropriately.  

It works and, while it’s not an example of excellent standards-compliant code, there was literally no other alternative available.  

Mobile overhaul

The previous code for the mobile version of THP was incredibly spotty. This is because it had a lot of exceptions for specific browsers and devices and it’s frankly a miracle that it worked as decently as it did. It was there as a stopgap because it was also unacceptable that THP didn’t have a better view for mobile users.  

All of what is the mobile-specific CSS was redone from scratch. As the board software was made during a time when the N-Gage was a thing, it hadn’t really been designed around being mobile-accessible. As such a lot of the inherent structure of how the boards themselves are written in the HTML is… well, pretty shitty. I’ve had to do a lot of tweaks and changes without breaking the whole structure in order to be able to hook in the things that mobile users would need to see things better on their devices.  

The end result, while not perfect, is that things should look way more consistent across devices and browsers and scale in a fashion that’s not too crazy. It may yet not break long words and symbols in all cases and it may get screwy at really high (like 2160p) or really low resolutions but it’s just impossible to get it any better without fundamentally altering how the data is structured on each page. For that I might as well do a complete overhaul of the site and implement more modern and scalable concepts every step of the way.

It may not seem like a lot of changes on the surface but this is probably one of the most time-consuming things I had to work on in the last few months. Every change had to be tested against and regressions often did crop up. It’s unlikely that I’ll be messing much more with this aspect of the site in the near future as it’s simply too much work to change any one thing, as small as it may seem.  

Other small stuff

I fixed an outstanding bug in the administration side of thigns that had been bothering me for a hwile. Basically board preferences wouldn’t be saved in some cases due to references to old bitso f code that no longer worked. Figuring out what those parts were and removing them, updating the SQL insertions to not account for them and the like took more time than I care to admit. As one of Kusaba X’s problems is that data isn’t structured in a concise way and there’s a lot of really large files that do too much at once, it’s slow going to figure out when something is broken.  

I also set different cache settings on the site. You can’t cache too much, like the HTML, otherwise you’re liable not to see when threads are updated. But loading things like CSS or scripts every time from the webserver makes page loading slower. So striking a balance is key. A lot of it depends on the server software as filtering by types of data isn’t as easy as it might sound. Also worth taking into consideration that different browsers deal with caches in different manners. It’s gotten less bad over the years but this is still something that took way too much time to figure out settings that would help most users.  

There’s a few other small things that were changed or had me fuming until I got them sorted. But I think I’ve covered the more interesting parts. I’ll likely talk about new things next time around, so look forward to that. Take it easy!