Building my website from scratch with Node.JS

Introduction

I made my first ever website in 2019, when I was 12, using wordpress.com, a basic CMS (Content Management System) tool that allows you to easily create a website without writing a single line of code. By designing your website with drag and drop widgets and using ready-made themes, you can make a website with wordpress.com very easily and quickly. It is also has a free basic plan which hosts your website on a wordpress.com subdomain allowing you to start making your website quickly. However, after a while, I became frustrated with the limitations of wordpress.com as it would not let me customise the website as much as I wanted to.

Migrating to Wordpress.org

I started researching alternatives to wordpress.com, which is when I discovered wordpress.org. wordpress.org is a more complex and flexible version of wordpress.com that can be installed on any server to host your website. An easy way to get started with wordpress.org is to pay a hosting provider to host a wordpress.org website for you; the three major wordpress.org hosting providers I looked at were SiteGround, Bluehost and DreamHost. I decided to use SiteGround because of its pricing which suited my very limited budget and the features available.

Setting up wordpress.org was quite interesting, as I learned about a lot of stuff such as Cloudflare, CDNs (Content Delivery Networks), FTP (File Transfer Protocol), SSH (Secure Shell), image optimisation using WebP, and much more. So it was a great learning experience for my web development knowledge.

The problem with wordpress.org

After about a year of using wordpress.org though, I eventually got frustrated with it as I thought I'd have more control over the code for my website. So I started researching how to build my own website from scratch using HTML, CSS and Javascript. When I started researching how to code my website, I knew very little about these languages. I knew I had a lot of learning ahead.

Making the website

Hosting

At the beginning of my project, I knew nothing about back-end website development, so I did not know where to start. The first thing I researched was how to get your website on the internet. I found out you needed to have a web server installed. The first one I considered using was the apache web server but after looking into it in more detail I decided to look at some alternatives as it did not allow as much customisation as I wanted and seemed quite complicated.

I then discovered the more customisable Node.js. Node.js is a JavaScript runtime that allows you to run JavaScript on a server like a regular programming language. Node.js has HTTP and HTTPS modules that can be used to make websites. After researching more about running a website on Node.JS, I found that most people use a Node.JS library called Express. Express is a highly minimal and flexible web framework that allows you to easily make a powerful website by setting custom HTTPS requests and responses.

I originally wanted to host my website on my raspberry pi, which is basically a small and powerful computer that can be used as a regular PC or a server. After looking into how to do this, I eventually found out that this would not work as my internet is nowhere near fast enough to host the website efficiently. There would also be a lot of security stuff I would have to do to make sure that my home network did not get hacked.

As a result of this, I started researching cloud hosting platforms. The three hosting vendors I considered were AWS (Amazon Web Services), Google Cloud and Microsoft Azure. After looking at the different features and pricing, I eventually chose AWS because it was a reasonable price for my budget; it was also very well documented, meaning it would be easier to work out how to do things and to troubleshoot errors, there were also many tutorials specifically on how to host a Node.Js website.

Setting up Visual Studio Code

After I'd worked out how I'd host the website, I started creating the frontend of the website (the HTML, CSS and JavaScript). I first set up Visual Studio Code to use as my programming software. I installed some helpful web development extensions such as Auto Close Tag, Auto Rename Tag, Bracket Pair Colorizer, Prettier and many more.

I also started using GitHub properly for this project, which is very helpful and fun to use. I set up the git/GitHub integrations with VS Code which would help me manage different versions and changes as well as provide a backup in case I lost my local code files. I am still pretty new to using git and GitHub, so I am still trying to work out how to use them properly.

Making the frontend

I spent some more time studying CSS and HTML so that I could build the homepage and the other pages from my old site. I aimed to develop just an MVP (Minimum Viable Product) before my old site's hosting annual subscription expired; I didn't want to have to spend another year of hosting when I didn't need to. So my deadline was set for December 2021.

I first recreated my old website homepage, which was very difficult to do as it used a lot of complicated CSS tricks to get all of the floating elements to be properly located on all devices. Once I had got all of the floating elements placed correctly, I then created the navbar (which was originally just a slightly modified version of a tutorial one from W3Schools) and the rest of the page, which was quite tricky to do as I still wasn't very good at HTML and CSS, but I eventually finished the homepage and the next few pages.

Keeping site-wide elements up to date across pages

After I had created the first few pages, I started to find that elements which were present on every single page, such as the top navbar and the footer, were hard to keep up to date. When you wanted to make any change to any of these elements, you needed to then change that element on every single page. As you add more and more pages, this would be very difficult and time-consuming to do; it would also be incredibly easy to miss a page or make a mistake.

When searching for solutions to this online, I found a potential solution to this problem on W3schools: HTML includes. Unlike CSS and JavaScript, HTML doesn't actually have a built-in includes/imports feature (An includes or imports feature in a language is used to add other files of code to the current script eg. including a module in javascript to run the code in the webpage). Although you can't just type an <include> tag to just add HTML code to the page (although this feature should definitely be added!), you can easily create the feature yourself using JavaScript, as shown on this page on W3Schools. The only problem with this solution is that every time you include something, it has to wait for the JavaScript file to load and then has to send an HTTP request to the server to get the HTML elements which was far too slow for the navbar, which was right at the top of the website, so I started trying to think of different solutions.

After trying a lot of different potential solutions to this problem, I eventually thought of an incredibly simple and easy way to do it. Instead of the original method where you would send an HTTP request and then have to properly insert the response into the page, you could instead just put the HTML code inside of a JavaScript file which would just automatically add the code to the page quickly. This method greatly increased performance and also fixed one of the biggest issues with building the site as well as being incredibly easy to do.

The reason that this works so well is that every single page can access that one JavaScript file meaning that you just need to update the one JavaScript file and it would automatically be updated on every single page. There are still two small issues with this method, the first being because this method relies on JavaScript if the user has JavaScript disabled in their browser, then it won't work. The second issue is if they are using some prehistoric browser that does not support it, it will not work and the elements that added with these scripts will be missing which is very bad because they are all very important elements (although this is not much of a problem because 100% of people with a working brain will not have disabled JavaScript and won't be using a browser that belongs in a museum!)

Building the backend of the website

Once I had finally made the MVP (Minimum Viable Product) of the website, I then started making the backend of the website. Currently, my website is only using web 1.0 (meaning it does not have any dynamic content/database) so I was hoping that there was not a lot of work to do on the backend. The backend of the website was just managed by one script which was using express JS for a basic webserver. The script is very simple but also has all the necessary security elements put in place. It did not even work with HTTPS yet (I didn't end up needing to anyway because of the way that I eventually set it up with AWS)!

Getting the website to run on AWS

After the basic components of my website were built, I started researching how to host it with AWS. I first found a helpful tutorial (I tried to find the webpage to link to it, but I literally could not find it again) that showed how to use Elastic Beanstalk (an AWS service) to host an express JS web app. Firstly, I set up my AWS account and committed the "working" version of my website to GitHub. The tutorial showed you how to setup AWS so that every time you updated the Github repository, it would then automatically update the version of the website on AWS which was very quick and helped save a lot of time! I then set up an environment in Elastic Beanstalk, which would host the website, and AWS CodePipeline, which is what automatically pulls website updates from GitHub, which I had a few issues trying to set up but I eventually got them both configured properly.

I then finally opened my new website and saw an error which probably meant hours of troubleshooting ahead, yay! First of all, I had to set express.JS to work with NGINX because I had forgotten to do that, which meant that the NGINX did not know how to properly send the traffic to the website and the website did not know how to accept the traffic. Once I had tried to fix that, the website still would not work :( so I started trying to find why the server was not properly responding to any requests.

After doing a bit more research, I found out that I could remotely connect to the Elastic Beanstalk server instance via SSH so that I could check that all the modules were installed properly and whether any errors were occurring which would be very helpful for troubleshooting the problem. After I had worked out how to SSH into the server (which took quite a while because it was very confusing!) and found where the website files were located (by using a Linux terminal command), I tried to use a command to run the Node.JS/JavaScript file but it told me that it was already running, so I looked at the node_modules folder (which is where node.JS modules such as express are stored) and I saw that some of the packages were missing and I also found that it was running a different version of node.JS than the one that I used which could be causing issues but also meant that the package.json/package-lock.json files were either incorrect or not working. Once I had fixed the two files so that they had the correct information, I pushed the changes to Github and it still did not work, but the node.JS version on the Elastic Beanstalk environment was now correct which meant I was closer to getting it working.

I then spent a while searching the web for tutorials on how to host an express.JS website on Elastic Beanstalk (which there aren't that many of because it's very specific) to try and find what was wrong with my code/Elastic Beanstalk set up and I finally found a tutorial on youtube where I saw that what you are supposed to do is use "process.env.PORT" for the webserver's port instead of manually choosing a port because AWS sets specific ports that you can use!

After I had done that, the website was finally up! But it was not attached to my domain name, "redstarbird.com", but was instead attached to some really long AWS subdomain so I still needed to get my domain name working and moved from my old web host to AWS; the website also only supported HTTP, not HTTPS, so I also needed to set that up and had basically no idea how to do either of these!

Setting up my domain name in AWS with Route 53

After looking up how to use a website domain with AWS Elastic Beanstalk, I found a tutorial from AWS (specifically this one) that shows you how to route traffic from AWS Route 53 to an Elastic Beanstalk environment. Route 53 is a DNS service which is on AWS which makes it easy to use it with any other AWS services so, following this tutorial, I moved my website domain from my previous DNS host to Route 53 which took 11 days which was pretty annoying but I think it was just because of the way that DNS works (possibly? I don't know too much about how DNS works) so I just had to wait for the domain name to finish transferring before I could do anything else, which was fine because I had other projects I wanted to work on in the meantime.

Once the domain had finally transferred to AWS, I followed the rest of the tutorial about routing traffic from the domain to Elastic Beanstalk, which was a little confusing to get working as I was not sure how to do a lot of the things on the tutorial but I eventually worked it out and my website was finally live and the only thing left to do was to get HTTPS working.

Setting up and forcing traffic to HTTPS connections

To set up HTTPS you need an SSL certificate, which encrypts all your traffic and proves that your site is actually secure somehow (I have basically no clue how it actually works but luckily I don't really need to, to be able to set up HTTPS on my website). I had previously tried to get an SSL certificate from AWS certificate manager (yet another AWS service) while my domain was still transferring but I needed to prove that I actually owned the domain I was trying to create the certificate for and it wasn't working while the domain was transferring :( but after the domain transfer had been completed, I finally managed to get the SSL certificate so that I could set up HTTPS, yay!

After looking through countless tutorials and posts on Stack Overflow and trying lots of different methods to set it up I finally found a simple method that just requires you to add a traffic listener to the load balancer on Elastic Beanstalk on port 443 (the standard port for HTTPS traffic) and select the SLL certificate to easily get HTTPS working! :) To make sure that all traffic to HTTP was rerouted to HTTPS, I went into the settings for the EC2 instance security groups (basically the security for which ports traffic can go to) and added an inbound rule that automatically redirected all traffic on port 80 (which is all HTTP traffic) and redirected it to HTTPS; I then disabled port 80 on the load balancer on Elastic Beanstalk to make sure that it would not send the webpage over HTTP before it could be redirected.

Conclusion

Overall I found this project quite difficult, mainly because of all the complications with getting AWS set up for my first time, but very enjoyable to work on as I learnt a lot and it was a fun challenge. When you make a Github commit titled like this:
It shows just how difficult a project is. I am now really pleased that I can have full control over my site because that is what I have always wanted to be able to do. I am also really pleased that my site homepage has this high of a score on the lighthouse website scanner tool!

What next?

Now that I have got my own site up and running, I'm looking forward to building a lot more features and functionality, such as a cookie banner (which I started trying to make but was very difficult to make from scratch and the OneTrust cookie banner slowed down my site a lot so I didn't want to use it), a database (which will require a lot of extra security), an e-commerce section, a subscription newsletter and much more. I also want to eventually be running the site off my raspberry pi using solar power which would allow me to host the website basically for free, although this would probably be quite challenging but also very fun to learn how!