This tutorial demonstrates how to build a web server using uWSGI, Nginx, Flask and Python and deploy into a docker container. The web server runs a SQLite database and provides a REST API to run SQL queries from an HTML website.
The focus in this tutorial will be on how to serve static files, setup the docker image and work with the REST API.
1. Prerequisites
- Docker and docker-compose (there are already plenty of tutorials about how to setup uWSGI and Nginx, so we use a predefined docker image which is already setup: https://github.com/tiangolo/uwsgi-nginx-flask-docker)
- SQLite is part of the python standard library since version 2.5. We use a Python 3.8 image, so a normal import will work.
2. Starting steps
- Install docker and docker-compose respectively
- Download the project from GitHub
- Open a terminal window and go into the folder of the unzipped project
- (Optional) Run ‘docker-compose -up’ or ‘sudo docker-compose -up’ as root if you have problems with the permissions to build and start the web server directly in the /uwsgi-nginx-flask-python-sqlite-docker-example/ folder
3. Project layout
This is how the Git project is structured.
└── uwsgi-nginx-flask-python-sqlite-docker-example ├── app │ ├── connection │ │ ├── __init__.py │ │ └── sqlite3_connection.py │ ├── database │ │ └── student.db │ ├── main.py │ ├── static │ │ ├── css │ │ │ └── style.css │ │ ├── favicon.ico │ │ ├── img │ │ │ └── tutorial-academy.png │ │ ├── index.html │ │ └── js │ │ └── func.js │ └── uwsgi.ini ├── docker-compose.override.yml ├── docker-compose.yml ├── Dockerfile ├── README.MD ├── LICENSE ├── .dockerignore └── .gitignore
The app directory contains three sub directories:
- Static contains our static web files we want to serve like index.html, images, style sheets or java script code.
- Database contains a predefined SQLite table example about student names and identification numbers
- Connection is a python module to encapsulate the SQLite database access
The .gitignore excludes files which do not belong into the Git repository, like temporary files, cache etc.
Similar to the .dockerignore file, which specifies directories and files that should not be copied into the docker container. This helps to keep the container as lightweight as possible.
3.1. SQLite database
The student.db is kept simple and consists of only two (three if you count the primary key) rows.
id | name | nr 1 | Mark Twain | 1111 2 | John Doe | 2222 3 | Harry Potter | 3333
3.2. Docker configuration
Let us start with the docker file:
FROM tiangolo/uwsgi-nginx-flask:python3.8 ENV STATIC_INDEX 1 COPY ./app /app
The first line selects the image for our container. The container is based on the tiangolo/uwsgi-nginx-flask:python3.8 image. It is a predefined image for uWSGI, Nginx and Python 3.8 usage. We use the ENV command to set the variable STATIC_INDEX to 1. Consequently, our /static/index.html will be served by requesting the root directory at http://127.0.0.1/. Finally, the app directory is copied into the container.
Have a look at the docker-compose.yml.
version: '3'
services:
web:
build: ./
In the docker-compose.override.yml, we define a container called web, specify a volume and port and finally the environment (e.g. the main flask app, and the run command for the start up).
version: '3'
services:
web:
volumes:
- ./app:/app
ports:
- "80:80"
environment:
- FLASK_APP=main.py
- FLASK_DEBUG=1
- 'RUN=flask run --host=0.0.0.0 --port=80'
3.3. uWSGI configuration
The last piece of configuration is the uwsig.ini.
[uwsgi] module = main callable = app
Generally, flask apps expose the ‘app’ callable instead of ‘application’ which we have to adapt here.
4. Flask web server
The main code of our web server.
import sqlite3
import json
import os
from flask import Flask, request, send_file, redirect, url_for
from connection.sqlite3_connection import Sqlite3Connection, sqlite3_call
app = Flask(__name__)
@app.route("/")
def main():
index_path = os.path.join(app.static_folder, "index.html")
return send_file(index_path)
@app.route('/favicon.ico')
def favicon():
return redirect(url_for('static', filename='favicon.ico'))
@app.route("/run", methods=["POST", "GET"])
def run():
"""
In this function the connection is open and closed with every call -> inefficient
"""
path = "./database/student.db"
query = ""
database = Sqlite3Connection(path)
database.open()
# extract query parameters
if request.method == "GET":
query = request.args.get("query")
elif request.method == "POST":
query = request.form["query"]
try:
result = sqlite3_call(database, query)
output = json.dumps(result)
except sqlite3.Error as err:
output = err.args[0]
database.close()
return output
if __name__ == "__main__":
# Only for debugging while developing and running main.py (without docker):
# -> choose a port higher than 1000 to avoid permission problems
#app.run(host="0.0.0.0", port=5000, debug=True)
# Port 80 configuration to run via docker-compose up
app.run(host="0.0.0.0", port=80, debug=True)
At first we define the flask app with several routes. The main route serves the index.html in debug mode (not required when running via docker due to the STATIC_INDEX = 1), and the favicon route to send the favicon.ico. In run we open the SQLite database, select between GET or POST query parameters and send the call to our database wrapper.
import sqlite3
class Sqlite3Connection:
"""
Sqlite3 wrapper class to open, close and access the connection
"""
path = None
conn = None
def __init__(self, path):
self.path = path
def open(self):
if self.conn is None:
try:
self.conn = sqlite3.connect(self.path)
except sqlite3.Error as e:
print(e)
def close(self):
if self.conn is not None:
self.conn.close()
def get(self, query):
if self.conn is None:
self.open()
try:
cur = self.conn.cursor()
cur.execute(query)
result = cur.fetchall()
except sqlite3.Error as err:
result = "Error - " + err.args[0]
self.close()
return result
def put(self, query):
if self.conn is None:
self.open()
try:
cur = self.conn.cursor()
cur.execute(query)
row_count = cur.rowcount
self.conn.commit()
response = "Done - Rows affected: " + str(row_count)
except sqlite3.Error as err:
response = "Error - " + err.args[0]
self.close()
return response
def sqlite3_call(database, query):
"""
Differentiate between SELECT and INSERT, UPDATE, DELETE
(hack -> wont work with sub-selects in e.g. update)
"""
if query == "" or query is None:
return "Warning: Empty query string!"
elif query and query.lower().find("select") >= 0:
return database.get(query)
else:
return database.put(query)
This wrapper has some error handling and a hack to differ SELECT (get) from INSERT, UPDATE and DELETE (put) queries. Check out the sqlite3_call function for that.
5. Result
Start the docker container with ‘docker-compose up’ or ‘sudo docker-compose up’ if you run into permission troubles. For security reasons you should not start via root. Define your own users and groups with corresponding permissions.
ta@ms:~/Developer/Workspace/Python/uwsgi-nginx-flask-python-sqlite-docker-example$ sudo docker-compose up Creating uwsgi-nginx-flask-python-sqlite-docker-example_web_1 ... done Attaching to uwsgi-nginx-flask-python-sqlite-docker-example_web_1 web_1 | Checking for script in /app/prestart.sh web_1 | There is no script /app/prestart.sh web_1 | /usr/lib/python2.7/dist-packages/supervisor/options.py:461: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security. web_1 | 'Supervisord is running as root and it is searching ' web_1 | 2020-09-08 20:18:41,922 CRIT Supervisor is running as root. Privileges were not dropped because no user is specified in the config file. If you intend to run as root, you can set user=root in the config file to avoid this message. web_1 | 2020-09-08 20:18:41,923 INFO Included extra file "/etc/supervisor/conf.d/supervisord.conf" during parsing web_1 | 2020-09-08 20:18:42,002 INFO RPC interface 'supervisor' initialized web_1 | 2020-09-08 20:18:42,002 CRIT Server 'unix_http_server' running without any HTTP authentication checking web_1 | 2020-09-08 20:18:42,004 INFO supervisord started with pid 1 web_1 | 2020-09-08 20:18:43,008 INFO spawned: 'nginx' with pid 9 web_1 | 2020-09-08 20:18:43,014 INFO spawned: 'uwsgi' with pid 10 web_1 | [uWSGI] getting INI configuration from /app/uwsgi.ini web_1 | [uWSGI] getting INI configuration from /etc/uwsgi/uwsgi.ini web_1 | web_1 | ;uWSGI instance configuration web_1 | [uwsgi] web_1 | cheaper = 2 web_1 | processes = 16 web_1 | ini = /app/uwsgi.ini web_1 | module = main web_1 | callable = app web_1 | ini = /etc/uwsgi/uwsgi.ini web_1 | socket = /tmp/uwsgi.sock web_1 | chown-socket = nginx:nginx web_1 | chmod-socket = 664 web_1 | hook-master-start = unix_signal:15 gracefully_kill_them_all web_1 | need-app = true web_1 | die-on-term = true web_1 | show-config = true web_1 | ;end of configuration web_1 | web_1 | *** Starting uWSGI 2.0.18 (64bit) on [Tue Sep 8 20:18:43 2020] *** web_1 | compiled with version: 8.3.0 on 09 May 2020 21:28:19 web_1 | os: Linux-5.4.0-45-generic #49-Ubuntu SMP Wed Aug 26 13:38:52 UTC 2020 web_1 | nodename: 513dced5fd72 web_1 | machine: x86_64 web_1 | clock source: unix web_1 | pcre jit disabled web_1 | detected number of CPU cores: 3 web_1 | current working directory: /app web_1 | detected binary path: /usr/local/bin/uwsgi web_1 | your memory page size is 4096 bytes web_1 | detected max file descriptor number: 1048576 web_1 | lock engine: pthread robust mutexes web_1 | thunder lock: disabled (you can enable it with --thunder-lock) web_1 | uwsgi socket 0 bound to UNIX address /tmp/uwsgi.sock fd 3 web_1 | uWSGI running as root, you can use --uid/--gid/--chroot options web_1 | *** WARNING: you are running uWSGI as root !!! (use the --uid flag) *** web_1 | Python version: 3.8.2 (default, Apr 23 2020, 14:22:33) [GCC 8.3.0] web_1 | *** Python threads support is disabled. You can enable it with --enable-threads *** web_1 | Python main interpreter initialized at 0x55fecea74320 web_1 | uWSGI running as root, you can use --uid/--gid/--chroot options web_1 | *** WARNING: you are running uWSGI as root !!! (use the --uid flag) *** web_1 | your server socket listen backlog is limited to 100 connections web_1 | your mercy for graceful operations on workers is 60 seconds web_1 | 2020-09-08 20:18:44,249 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) web_1 | 2020-09-08 20:18:44,249 INFO success: uwsgi entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) web_1 | mapped 1239640 bytes (1210 KB) for 16 cores web_1 | *** Operational MODE: preforking *** web_1 | WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0x55fecea74320 pid: 10 (default app) web_1 | uWSGI running as root, you can use --uid/--gid/--chroot options web_1 | *** WARNING: you are running uWSGI as root !!! (use the --uid flag) *** web_1 | *** uWSGI is running in multiple interpreter mode *** web_1 | spawned uWSGI master process (pid: 10) web_1 | spawned uWSGI worker 1 (pid: 13, cores: 1) web_1 | spawned uWSGI worker 2 (pid: 14, cores: 1) web_1 | running "unix_signal:15 gracefully_kill_them_all" (master-start)...
The container and the web server started successfully. Subsequently go to http://127.0.0.1/ and check out the result.
Click on the ‘Select all’ button and click the ‘Run query’ button afterwards. You should see some JSON results or error messages in a pop-up message.
You can try the SELECT, INSERT, UPDATE or DELETE options on the right or just enter your own queries. Have a look at the output from each request in the docker container. Sub selects should not work because of the hack from above. We have not tried to use CREATE or ALTER Table commands etc. Feel free to do so!
6. Conclusion
In this tutorial we learned about the configuration files and how to serve static files in a dockerized application. There always is a trade-off between prepared docker images and building up images by yourself. While you can probably simplify and reduce the size or libraries etc. in one image, it will take more time for you to do so.
For the purpose of demonstration, we tried to keep it as simple as possible.
Working with docker containers and building micro services or REST applications is a great starting point to think about software architecture and its benefits. Future development of big and complex software will shift into that direction.
While the initial effort for setting up containerized software is higher, the return of invest arrives after a while due to faster or more flexibel development, better maintenance, scalability or running faster updates.
Being independent from the implementation of the micro services offers further opportunities for new colleagues, external service providers and many more.
If you have problems or questions feel free to ask.
