Deploying

Here we explain how to bootstrap a new ColDoc web portal, or site.

We assume that the ColDoc source code was downloaded in the directory whose name is saved in the environment variable COLDOC_SRC_ROOT. For details see the install section

ColDoc keeps a strict separation between code and data.

The same code directory can be used to run many sites.

In turn, each site can host many documents.

In the install section we installed the code. Here we will prepare the structure for the data and settings for a portal.

To start a new ColDoc site, you need to setup a directory containing some files. This process is called deploying. The directory name must be saved in the COLDOC_SITE_ROOT environmental variable.

You need to use a terminal where you can insert shell commands.

Serving with Apache

To serve the portal using Apache2 in Debian or Ubuntu, you may install the packages

sudo apt install apache2 libapache2-mod-wsgi-py3

It is advisable to put the portal under /var/www (or otherwise, you should edit /etc/apache2/apache2.conf otherwise apache will not serve your content). Here is an example shell code:

export COLDOC_SITE_ROOT=/var/www/test_site
sudo mkdir ${COLDOC_SITE_ROOT}
sudo chown owner.group ${COLDOC_SITE_ROOT}

where owner.group is who is performing the install.

Serving without Apache

If you want to run the portal by some other means (there are many ways to deploy Django, see here ) then you may setup the test site anywhere, let's say /home/.../test_site . Make sure that this directory is empty, and set its name in an environ variable as follows.

export COLDOC_SITE_ROOT=/home/.../test_site
mkdir ${COLDOC_SITE_ROOT}

Deploying the skeleton

In the following you may omit the part ${COLDOC_SRC_ROOT}/ if you are sure that the current working directory of the shell is the directory where the ColDoc source code is located.

This command will create the structure for the new ColDoc portal

python3 ${COLDOC_SRC_ROOT}/ColDocDjango/helper.py  deploy --coldoc-site-root=${COLDOC_SITE_ROOT}

In particular it will deploy the config file for the new document as ${COLDOC_SITE_ROOT}/config.ini. This contains some fundamental settings for the site, and it can also be used to activate/deactivate special features for the portal, such as: social authentication, background tasks, comments, etc. Edit it at taste.

Using MySQL

By default, the above command will create a portal that uses sqlite as backend database. If instead you use

python3 ${COLDOC_SRC_ROOT}/ColDocDjango/helper.py  deploy  --coldoc-site-root=${COLDOC_SITE_ROOT} --database=mysql

then the portal will be prepared for using mysql as database. In this case, you should create the database, the user and set permissions, before proceeding. You may use this command:

# sudo mysql < ${COLDOC_SITE_ROOT}/mysql.sql

If you deploy using sqlite and then change your mind later, there are instructions in the section issues section.

Local variables

There are many settings for a Django portal (the config.ini file will setup only the most important ones).

At startup, Django reads a settings.py file. In this case, settings for a deployed site are read from three files:

  • the general file ${COLDOC_SRC_ROOT}/ColDocDjango/ColDocDjango/settings.py in the ColDoc code;

  • ${COLDOC_SRC_ROOT}/ColDocDjango/settings_local.py if it exists;

  • ${COLDOC_SITE_ROOT}/settings.py from the COLDOC_SITE_ROOT directory where the web site is deployed.

Each one overrides the previous.

The last file is prepopulated with some useful examples (all commented out).

To better test the code, you may want to create a file ${COLDOC_SRC_ROOT}/ColDocDjango/settings_local.py to setup some variables to enable email sending, as in this example.

MAIL_HOST = "smtp.server"
EMAIL_PORT = "587"
EMAIL_HOST_USER = "username"
EMAIL_HOST_PASSWORD = "password"
EMAIL_USE_TLS = True
DEFAULT_FROM_EMAIL = "Helpdesk <helpdesk@that_email>"

or to enhance the code, e.g. adding some mimetypes used in your coldoc s

import mimetypes
# https://bugs.freedesktop.org/show_bug.cgi?id=5455
for j in ('.gplt','.gnuplot'):
    mimetypes.add_type('application/x-gnuplot',j)

Or you may want to enable them in ${COLDOC_SITE_ROOT}/settings.py for your specific site.

See in ${COLDOC_SRC_ROOT}/ColDocDjango/settings_suggested.py for more examples.

Social auth

If you wish to use social authentication, you may set use_allauth to True in ${COLDOC_SITE_ROOT}/config.ini and install django-allauth

Note that once you set `use_allauth` to True, you cannot change it back to `False`.

In particular, you may add stanzas for django-allauth in ${COLDOC_SITE_ROOT}/settings.py such as providers and configs, something like

INSTALLED_APPS += [
        'allauth.socialaccount.providers.google']
SOCIALACCOUNT_PROVIDERS = {
    'google': {
        'SCOPE': [
            'profile',
            'email',
        ],
        'AUTH_PARAMS': {
            'access_type': 'online',
        }
    }
}

and don't forget to connect to the admin interface and to create a social application in the database, that contains all credentials (in the above case, for Google OAuth2).

See django-allauth docs for more details

Moreover you may need to setup the Django smtp machinery, to send emails (emails are sent automatically to verify emails addresses or reset passwords).

Late adding of social auth

If you did not turn social authentication on at first, you may turn it on later, by following the above instructions; and then you have to run

python3 ${COLDOC_SRC_ROOT}/ColDocDjango/manage.py migrate
python3 ${COLDOC_SRC_ROOT}/ColDocDjango/manage.py collectstatic

to update the databases.

Initalize

Then initialize django for your deployed site

python3 ${COLDOC_SRC_ROOT}/ColDocDjango/manage.py migrate
python3 ${COLDOC_SRC_ROOT}/ColDocDjango/manage.py set_site [name] [url]
python3 ${COLDOC_SRC_ROOT}/ColDocDjango/manage.py collectstatic

where you substitute [name] and [url] for appropriate values for your portal.

Add test material

To test the portal we may populate it with the test LaTeX document.

Before we create some fake users, to be able to interact with the portal.

python3 ${COLDOC_SRC_ROOT}/ColDocDjango/helper.py  create_fake_users

(The list of users and passwords will be printed on terminal)

We insert the test LaTeX document in the portal. Note that jsmith is the author of all blobs, and will have special access rights; similarly ed_itor is the editor, and will have access to some administrative information in the coldoc main page.

python3 ${COLDOC_SRC_ROOT}/ColDocDjango/blob_inator.py --coldoc-nick=test --lang=eng --ZS --editor=ed_itor --author=jsmith  --SP --SAT    ${COLDOC_SRC_ROOT}/test/paper/paper.tex

Then you should generate all PDF and HTML associated to the test paper

COLDOC_URL="http://localhost:8000/UUID/test/"
python3 ${COLDOC_SRC_ROOT}/ColDocDjango/latex.py --coldoc-nick=test --url-UUID=${COLDOC_URL}  all

(The command line option --url-UUID is needed so that the hyperlinks inside the PDF version will point to the correct URL)

Activate the Apache portal

If you are preparing the web site to be served by Apache2, you should

sudo chown -R www-data:www-data ${COLDOC_SITE_ROOT}

otherwise Apache will not be able to access it. Then set up Apache as follows:

sudo cp ${COLDOC_SITE_ROOT}/apache2.conf /etc/apache2/sites-available/test_site.conf
sudo a2ensite test_site
sudo a2enmod wsgi
sudo systemctl reload apache2

To enjoy advanced caching capabilities you may also

sudo a2enmod expires
sudo a2enmod headers

If you activated whitenoise you may also tweak caching timings, as explained in apache2.conf .

Serve without Apache

Start the simplest Django server and access the portal

python3 ${COLDOC_SRC_ROOT}/ColDocDjango/manage.py  runserver 8000
firefox http://localhost:8000/

Note that in this case django will not serve the static files, unless you set debug to True in ${COLDOC_SITE_ROOT}/config.ini ; and you may need to change

dedup_root = %(site_root)s/static_local/dedup
dedup_url = /static/dedup

in that file.

Software upgrade and/or template changes

Note that each time you upgrade the software you need to

python3 ${COLDOC_SRC_ROOT}/ColDocDjango/manage.py migrate
python3 ${COLDOC_SRC_ROOT}/ColDocDjango/manage.py collectstatic

This is particularly important when you use whitenoise otherwise the cache will not work and your server will return 500.

Final remarks

ColDoc keeps a strict separation between code and data. You may even install the code using an account, let's say coldoc_sw, then deploy a portal, and assign all the files in the portal to a different user, let's say coldoc_data: in this case you need to tell Apache about this change, by adding the user and group directives in the line starting as WSGIDaemonProcess, as follows

WSGIDaemonProcess coldoc.group python-home=/...virtualenv.... python-path=${coldoc_src_root}  locale=en_US.UTF-8  lang=en_US.UTF-8 user=coldoc_data group=coldoc_data

This may improve security.

Customize templates

If you uncomment the line

TEMPLATES[0]['DIRS'].insert(0,os.path.join(COLDOC_SITE_ROOT, 'templates'))

in ${COLDOC_SITE_ROOT}/settings.py then you may use that directory to install templates that will override the shipped ones: see following sections for some useful examples.

Analytics

ColDocDjango/templates/analytics.html containes code to use Google Analytics 4, that will be activated simply by defining (in ${COLDOC_SITE_ROOT}/settings.py) the variable GOOGLE_ANALYTICS4 to contain the the correct key.

If you wish to use another analytics method, create the file ${COLDOC_SITE_ROOT}/templates/analytics.html and fill it with the needed code, and this will override the above template.

Extra header

Any other extra information you want to include in the html headers may be put into ${COLDOC_SITE_ROOT}/templates/extrahead.html --- as for example, favicon, apple-touch-icon and so on.