At some point you become paranoid

[steven36]

The Internet breaks your dreams

 

 

One day I found a snippet of code online that would randomly open the CD tray on my computer. When I opened the VBScript file, I was amazed how few lines of coded were needed to do this. I quickly grew interest to be the author of my own fun little programs. When I discovered programming I figured that with enough time, I could make the computer do whatever I wanted. I fell in love instantly. However, time has taught me that love is not what you feel when everything is alright. Love is surviving in the harshest of conditions. And surely, my love for programming has been tested when someone other then I, started using my little programs.

 

The first website I built seriously was a tutorial website. I needed a job, and my portfolio was pitiful. I created a web space where I listed links to all tutorials I read that taught me how to program. Everyday I would scour the web and find the best instructors and feature them on my humble little space. I was the only contributor at first but quickly, Google started to bring people in.

 

I created a little section where anyone could submit their own tutorials. There was a little description that explained how it was our duty as generous human beings to share knowledge and help others. I published the page and waited enthusiastically for people to add new tutorials. It didn't take very long for the first interaction to occur.

 

Opening my admin, I was greeted with a little star on the top left that notified me that a brand new tutorial was added to the website. I clicked on it and to my surprise, there it was. Someone had made a submission. But it was not a tutorial. It was just a message that said, "thank you for the great collection of tutorials."

 

Very nice of them, but this was not quite what this tool was designed for. This page was built for the sole purpose of adding tutorials. My email was plastered all over the website, he could have just sent me this thank you note on my email. But it gave me an idea. I went back to the drawing board and designed a new feature for the website. A commenting system.

 

Now if people felt the need to express themselves, wanted to leave a comment about a tutorial or say thank you, they could do it. But the Internet being the Internet, this is not how my users used the comment system.

 

One person decided to give scores to tutorials. He would read a tutorial and score it on a scale of 1 to 10. Other people followed his lead and rated things in a very inconsistent manner. That frustrated me. Some rated it on a scale of 1 to 5, some gave it a thumbs up, and one guy gave a tutorial a B. This gave me an idea for a new feature. I built a rating system.

 

With the shiny new rating system in place, no one would use the comment section to rate, instead they could just click on the little stars and that was that. Not surprisingly, people started rating tutorials.

 

But, the comment system was no longer being used. It took a while before one more person decided to take another look at it. And he used it in absolutely the wrong way. Of course a flock of users followed his lead. He started listing alternative tutorials right in the comment section. That's not what the comment section was for, the whole point of the website was for them to be able to send tutorials. There is a dedicated page just for that.

 

So I wrote in big letters to use the tutorial submission page to send tutorials. It was as if people were not reading and doing whatever they wanted. It would get me angry. And some people started sending hundreds of tutorials in a row, with no description or title. Looking at the time stamp, it seemed like someone had created a bot to submit tutorials on my website. I didn't want to stop whoever it was from submitting tutorials. So it gave me a new idea.

 

When you send a link, and there are no titles or description, I would use the URL to scrape the website to gather the information myself. It was easy to get the title of the page, but because of the inconsistency of HTML structure, I had to create a complex algorithm to extract content. It wasn't always right but it was better than nothing.

 

This was working out pretty well until the bots started sending things that were not URL. My script would fail because it was expecting only a URL. Now my website was filled with nonsense and it was increasing at an alarming rate. On the backend, there was no way to differentiate which content was submitted by a person and which by a bot. After meditating for an hour I took drastic measures to solve the problem.

 

I shutdown the website for a full day and created a new feature. No more would bots be allowed to send tutorials. Instead, a human would have to solve a captcha in order to submit anything. I built my own captcha using imagemagick. The amount of new content drastically went down, but I chose quality over quantity.

 

However, something new happened. Something I had never dealt with before. Spam.

 

The bots that submitted tutorials where designed specifically for my website. After all there was a human behind it. But not spambots. SpamBots are a larger network of machines that are designed to spam any website with a form. And they targeted my comment system. I'd get over five thousand new entries in a single day and it would drastically slow my amateur designed comment schema.

 

An important lesson in programming that was repeated over and over in the tutorials on my website was the DRY Principle. Don't Repeat Yourself. I had created a captcha for tutorial submission, so it was only a matter of copying it to the comment section. When I copied it, all spam stopped immediately. I went back to my admin to delete the massive amount of spam and that was the end of it. Only it wasn't.

 

Spam had pushed away my real users. Every once in a while I would see a comment that would say "Thanks" or "nice!", and then back to silence. This little place that started as a portfolio website to get me a real job had fulfilled its purpose, I now had a job. Little by little, I stopped paying any attention to it. I was no longer monitoring newly submitted tutorials. Slowly, I moved on.

 

6 months into the job, I decided to go back to my humble little place to see what people were now doing. When I typed the URL, it took over a minute before anything rendered on the page. When it finally did, my jaw dropped. I was at work and didn't want to get fired for visiting this type of website. I closed it immediately.

 

When I got home, I took another look. Every single thing had been abused. It was filled with spammy ads, the titles of every tutorial contained "viagra", and more nasty stuff. Some page redirected to other websites because of javascript code injected directly onto the page. The database was extremely slow, and I didn't know how to segregate the good content from the bad content. I still don't know why Google didn't drop me from search results entirely. So instead, I shutdown the website.

 

Believe it or not, this website was coded entirely on the production server. For the first time, I copied the website locally on my machine. I tested the captcha to see if was still working. It was, but I decided to make it a little more complex just to see. Every night after work, I'd spent time filtering spam from real content. It took weeks before I was confident enough to redeployed the cleaned up version back to the web. The homepage looked clean alright. I refreshed the page one more time to be sure, and chaos followed. In just a few seconds the spam was back.

 

I was updating the captcha code live on the server, and it was defeated the moment I saved it. The SpamBots were going through my captcha like butter. I tried for an hour and nothing to do. I threw in the towel. I disabled new comments, and new tutorial submission. In my 6 months absence I had lost all of my users anyway.

 

I left the website as it was with no option to add new content. When my hosting fees were due, I didn't pay. And the website disappeared into oblivion.

 

Many developers will tell you about how amazing it is to build websites. What they leave out is how tedious it becomes to maintain them. It is one thing to access your webpage when you know where everything is and what all the features are. It is a whole other to watch someone stumble on your badly designed UI.

 

At some point you become paranoid and build features to prevent the user from doing stuff you don't want them to. A common example is building a Hell banning system to block users who are annoying before you even start having users on the website.

 

Building my own little website may have been a source of frustration, but it was still an environment where I could experience the realities of the web first hand. You can always read about how bad cross site scripting is, but experiencing it on your own website and struggling to find solutions is a completely different experience.

 

I threw in the towel for that website, but anything I built after was be in light of this failure. I now know that user data is unpredictable, I know that security is not a feature but a necessity, I know that what I think is common sense is common only to me. It may turn web development into more than just writing code, but if that's all it was then our jobs would have been long replaced by robots.

 

I take this as a valuable lesson. But I can't help but be paranoid anytime someone asks me to build a new website.

 

The Source