Project Guide: Site

 GitHub Repo  Documentation

Environment

When contributing to the site, the things required in your environment vary by what exactly it is you're working on. Working on the API and working on page styling are going to require different sets of requirements. That said, there are some basic requirements that you must have set up before you can work on the site.

Setting up

First of all, make sure you have already met all the requirements from the Getting Started page.

Next, head over to the GitHub repo, and click on the "Fork" button to create a copy of the repository under your account.

Open a terminal (or Git Bash on Windows) and run the following, replacing YOUR_USERNAME with your GitHub username:

git clone git@github.com:YOUR_USERNAME/site.git

This will download a copy of the repository, ready for you to work on. Next, change directory into the repository folder and use Pipenv to install a virtualenv with all of the project requirements and tools:

cd site/
pipenv sync --dev

Create a file named .env and add the following to it:

BOT_API_KEY="test"
FLASK_DEBUG=1
TEMPLATES_AUTO_RELOAD=1

Before running the server, you will need to edit your hosts file. On Windows, you will need to do this by running Notepad as admin - see the link for a full guide. Add the following to your hosts file:

127.0.0.1    pythondiscord.local
127.0.0.1    api.pythondiscord.local
127.0.0.1    staff.pythondiscord.local
127.0.0.1    ws.pythondiscord.local
127.0.0.1    wiki.pythondiscord.local

Required services

You'll need RabbitMQ and RethinkDB to be set up before you can do any work on the site.

There are two ways of setting these up: Option 1 is using docker-compose (which is much easier, so please do this), and option 2 is manually installing, configuring and running the services on your computer.

Option 1:

  • Docker Compose
    • Find the documentation for installing docker

    • Find the documentation for installing docker-compose

    • From the root directory of the repository run the command: docker-compose up -d and wait for the containers to be downloaded and started. If you installed docker-compose on Linux using the recommended method, you will have to run it as root using sudo.

    • On Windows, these services will not be listening on localhost. You will need to figure out which IP they are listening on by looking at the log output produced by docker-compose up, note the lack of the -d flag. The log should say something like pdrdb_1 | Listening on cluster addresses: 127.0.0.1, 172.17.0.1, 192.168.65.3, 10.0.75.2. Try pinging these IPs to see if you can get through to any of them. If any of them respond, you should change your rethinkdb and rabbitmq hosts to this IP. This can be done by changing the environment variables RABBITMQ_HOST and RETHINKDB_HOST. You change these by adding them to the .env file we created in the previous step.

    • On Linux, once the containers are running, they will be listening on localhost. Simply use netstat -plnt | grep <port> to confirm that port 5672 and port 28015 are listening.

    • If rethinkdb or rabbitmq are already installed on your system, make sure they are stopped or uninstalled to avoid conflicting ports.

    • To stop the containers again, simply do docker-compose stop

Option 2:

  • Manual install of RethinkDB
    • On Linux, you'll want to find a package for your distro - please note that on Ubuntu 18, you'll have to download and install rethink from their latest release on GitHub instead. You'll want the file called rethinkdb_2.3.6.srh.1.0bionic_amd64.deb.

    • On Mac OSX, please install rethinkdb using Homebrew

    • On Windows, you can download RethinkDB here, just extract it somewhere and you'll be able to run it directly whenever you need it - we recommend that you add it to your PATH for ease of use

    • You should change the http_port for RethinkDB to something other than 8080, since the site will be running on that port. See the RethinkDB docs on config files for information on how to do this.

  • Manual install of RabbitMQ
    • On Linux, all you have to do is install the package for your distro and start it up - there is no configuration to do

    • On Mac OSX, please install rabbitmq using Homebrew - for more information, see the RabbitMQ for Mac page

    • On Windows, you can download RabbitMQ here

    • It is not necessary to add an account or change any passwords - the default credentials of guest/guest will work just fine

    • The site will start up without RabbitMQ, but it'll take some time to start up properly without it

Linting and Continuous Integration

On all our projects, we enforce code linting. This means that the code has to meet certain best practices and style requirements before it passes our automatic inspection systems, and can be merged into our codebase. We recommend you set up a linter for JS, SCSS and Python if you want to be able to work on the entire site. You should also use a git pre-commit hook to execute these before every commit. Our Continuous Integration systems runs these linters on all commits, and you can look at your pipeline on GitHub to see exactly what happens. If any of these linters fail, your commits will not be possible to merge into our master branch, so it's best to catch these errors before you commit.

In this section we will show you how to set up the linters and the hook.

SCSS

This is an optional requirement for those of you that are working with the SCSS files, which are used to style the site. If you're working with these files then we strongly suggest that you lint the files with ruby-scss-lint in order to maintain a consistent code style. To install ruby-scss-lint:

  • On Linux, you're likely to have a package named ruby-scss_lint, so just install that and you're good to go - If not, try to avoid any packages that mention Dart or JavaScript
    • If you're unable to find any packages like this, install Ruby and use gem install scss_lint instead

  • On Mac OSX, please install ruby and brew-gem using Homebrew, and then install the scss_lint gem using brew gem install scss_lint
    • That's all there is to it - No extra steps

  • On Windows, install Ruby using Ruby Installer (making sure you check the Add Ruby executables to your PATH box), then open an administrator command prompt and run gem install scss_lint
    • If you're on Windows 8 or later, you can open the command prompt this way by typing cmd into the start menu, right-clicking cmd in the list and selecting Run as Administrator

Once you've done all of the above, the following commands will become available to you:

  • pipenv run buildscss: Compile the SCSS files into CSS

  • pipenv run lintscss: Lint the SCSS files

JavaScript

This is an optional requirement for those of you that are working with the JavaScript files, which are used to provide additional functionality to users on the site. If you're working with these files, then we strongly suggest that you lint the files with eslint in order to maintain a consistent code style.

Additionally, we require that all JavaScript files are combined and minified, in order to keep things snappy for mobile users and users on slow connections. This is done using gulp.

To install these requirements:

  • Make sure you have a working install of Node.JS and that npm is available to run in a terminal on your system
    • On Mac OSX, you can use Homebrew: brew install node

    • On Linux, please check your package manager first

  • Install gulp-cli and eslint globally using npm
    • On Windows or Mac OSX: Open a command prompt or terminal and use npm install -g eslint gulp-cli

    • On Linux, gulp-cli and eslint may be available in your package manager, so check there first

  • Install gulp and its dependencies within your project directory
    • Open a terminal within the project folder, and use npm install to install gulp and its dependencies

Once you've done all of the above, the following commands will become available to you:

  • pipenv run buildjs: Combine and minify the JavaScript

  • pipenv run fixjs: Lint the JavaScript and fix some common problems

  • pipenv run lintjs: Lint the JavaScript without applying any automatic fixes

Python

We use the flake8 linter for Python, but this is installed by pipenv earlier in this guide when you do pipenv sync --dev, so you do not need to install it manually.

Git hook

If you'd like to automatically lint your code every time you commit to Git, you could make a pre-commit hook! Simply go into the hidden folder in your git directory called .git, open the hooks folder, and create a new file called pre-commit. Inside this file, you can put whatever you like. If at any point this file exits with exit code 1, the commit will be aborted! This means that we can run all the linters, and if any of them fail, we will return exit code 1. Here is an example script for doing just that:

#!/bin/bash

# Making sure we have pipenv installed
echo "-- Searching for pipenv --"

if ! hash pipenv 2>/dev/null; then
    echo "-- Could not find pipenv! Please install pipenv first. --"
    exit 1
fi

# Linting
pass=true

# Python linter
printf "\n\n-- Linting the Python using flake8 --"
flake8=$(flake8)
flake8_code="$?"
if [ "$flake8_code" != 0 ]; then
    echo "$flake8\n"
    pass=false
    echo "-- Python linter failed! --"
else
    echo "-- Python linter completed successfully --"
fi

# SCSS Linter
printf "\n\n-- Linting the SCSS using scss_lint --"
scss_lint=$(pipenv run lintscss)
scss_code="$?"
if [ "$scss_code" != 0 ]; then
    echo "$scss_lint\n"
    pass=false
    echo "-- SCSS linter failed --"
else
    echo "-- SCSS linter completed successfully --"
fi

# JS Linter
printf "\n\n-- Linting the JS with gulp --"
js_lint=$(pipenv run lintjs)
js_code="$?"
if [ "$js_code" != 0 ]; then
    echo "$js_lint\n"
    pass=false
    echo "-- JS linter failed --"
else
    echo "-- JS linter completed successfully --"
fi

# Did any of the linters fail?
if [ "$pass" == false ]; then
    printf "\n\n-- One or more of the linters failed. Please fix all linting errors and try again. Commit will be aborted. --\n"
    exit 1
else
    printf "\n\n-- All linters completed successfully. Commit approved."
fi

Working with the code

Make your changes in your favourite editor or IDE, and make sure you test your code. You can run a linter over the Python code and start the server with:

pipenv run lint  # Check the code for styling errors
pipenv run rundev  # Run the server for testing

By default, the server will run on port 8080 - you should be able to reach it at using http://pythondiscord.local:8080 for testing, if you've modified your hosts file as described above.

If you're working on the site styling by modifying the SCSS files, you can lint them and compile them with:

pipenv run lintscss  # Check the SCSS for styling errors
pipenv run buildscss  # Compile the SCSS into CSS

Likewise, if you're working with the JavaScript files, you can lint, combine and minify them with:

pipenv run lintjs  # Check the JS for styling errors
pipenv run buildjs  # Combine and minify the JS into one file

When you recompile the JS or SCSS, you may need to clear the cache in your browser to see the changes.

For more information on environment variables and other specifics of the code, please see the documentation page.