Code, Explained

Reusable Python Library for KML/KMZ Route Parsing and Google Maps URL Generation

Working with GPS routes often means dealing with KML and KMZ files exported from Google Earth, Garmin devices, route planners, or GIS applications. While these formats are widely used, extracting route information and converting it into something practical—such as a Google Maps directions link—usually requires manual effort.

To simplify this process, I built a Python solution that consists of two parts:

  1. A reusable library class that can be imported into any Python application.
  2. A command-line utility that uses the library for interactive and automated workflows.

The result is a flexible tool that can parse KML and KMZ files, generate Google Maps URLs, and export location data to CSV.

Github: https://github.com/razonklnbd/kml-to-google-map-url

Why I Built This Project

This project started from a real family travel planning problem.

I was planning a long road trip for my family using Google My Maps. My goal was to create a custom route with multiple stops and then print step-by-step driving directions for my daughter so she could easily follow the journey and understand the route ahead of time.

Google My Maps is excellent for planning and visualizing custom routes, but I quickly discovered a limitation: it does not provide the same printable turn-by-turn directions that are available in the standard Google Maps interface.

One option was to manually recreate the route in Google Maps by entering every stop one by one. While this works for a route with only a few locations, it becomes tedious and time-consuming when dealing with dozens of waypoints spread across a multi-day trip.

Since Google My Maps allows routes to be exported as KML files, I started looking for a way to automatically convert that route data into a Google Maps Directions URL that could be opened directly in the standard Google Maps interface.

As a Python developer, I realized this problem was straightforward to automate:

  1. Export the route from Google My Maps as a KML or KMZ file.
  2. Extract all placemark locations.
  3. Generate a Google Maps Directions URL using those locations.
  4. Open the generated URL in Google Maps.
  5. Print the step-by-step directions from Google Maps.

What began as a small utility script for a family vacation quickly evolved into a reusable Python library and command-line tool that can parse KML/KMZ files, generate Google Maps route URLs, and export route information to CSV.

The result is a tool that saves time, eliminates manual data entry, and makes it easy to move route data from Google My Maps into standard Google Maps for navigation and printing.


Project Goals

The design goals were:

  • Support both KML and KMZ formats
  • Generate Google Maps directions URLs
  • Export route points to CSV
  • Provide both GUI and command-line workflows
  • Separate business logic from user interface code
  • Allow other developers to reuse the parsing functionality in their own projects

Architecture Overview

The project is divided into two files.

1. Library Layer

kml_route_parser.py

Contains:

class KmlRouteParser

Responsibilities:

  • Read KML files
  • Read KMZ files
  • Extract placemark points
  • Generate Google Maps URLs
  • Export CSV files
  • Provide route data programmatically

The library contains no user interface code.


2. Application Layer

main.py

Responsibilities:

  • Command-line argument processing
  • File selection dialogs
  • Save dialogs
  • User prompts
  • Displaying results

This separation follows a clean architecture approach where the reusable logic is independent of the user interface.


Supported File Formats

KML

KML (Keyhole Markup Language) is an XML-based format used to store geographic data.

Example:

<Placemark>
    <name>Home</name>
    <Point>
        <coordinates>
            72.5432,23.0345,0
        </coordinates>
    </Point>
</Placemark>

KMZ

KMZ is a compressed ZIP archive containing one or more KML files.

The library automatically:

  • Detects KMZ files
  • Extracts the embedded KML
  • Parses the route data
  • Cleans up temporary files

The user does not need to manually extract anything.


Extracting Route Points

Each placemark is converted into a structured Python dictionary.

Example:

{
    "name": "Home",
    "description": "Starting Point",
    "latitude": 23.0345,
    "longitude": 72.5432
}

The parser returns a list of points that can be consumed by any Python application.


Google Maps URL Generation

One of the primary features of the library is automatic generation of Google Maps Directions URLs.

The library supports two modes.


Name-Based URLs

Example:

https://www.google.com/maps/dir/Home/Office/Warehouse

Usage:

url = parser.get_google_maps_url(
    use_names=True
)

This produces human-readable URLs.


Coordinate-Based URLs

Example:

https://www.google.com/maps/dir/23.0345,72.5432/23.0654,72.6021

Usage:

url = parser.get_google_maps_url(
    use_names=False
)

Coordinates are often more reliable because they eliminate ambiguity caused by duplicate place names.


CSV Export

The library can export extracted points into a CSV file.

Example:

Name,Latitude,Longitude,Description
Home,23.0345,72.5432,Starting Point
Office,23.0654,72.6021,Destination

Usage:

parser.save_csv(
    "points.csv"
)

This allows route data to be easily opened in:

  • Excel
  • Google Sheets
  • Power BI
  • GIS software
  • Data analysis tools

Using the Library in Your Own Projects

The biggest advantage of the refactoring is that developers can now use the parser without the command-line application.

Example:

from kml_route_parser import KmlRouteParser

parser = KmlRouteParser(
    "route.kmz"
)

parser.load()

url = parser.get_google_maps_url(
    use_names=False
)

print(url)

Accessing Route Points Programmatically

You can access all extracted locations directly.

Example:

points = parser.get_points()

for point in points:

    print(
        point["name"],
        point["latitude"],
        point["longitude"]
    )

This makes the library suitable for:

  • GIS applications
  • Fleet management systems
  • Route optimization tools
  • Data processing pipelines
  • Custom map applications

Command-Line Utility

The project also includes a command-line application built on top of the library.

File:

main.py

Interactive File Selection

When no input file is supplied:

python main.py

a file picker dialog appears.

Supported file types:

  • KML
  • KMZ

This makes the tool accessible to non-technical users.


Command-Line File Input

Files can also be provided directly.

Example:

python main.py route.kml

or

python main.py route.kmz

This mode is useful for automation.


URL Mode Selection

If URL mode is not supplied through the command line, the application asks the user:

Build URL using:

1. Place Names
2. Coordinates

The selected mode is passed to the library.


Automated URL Mode

URL mode can be specified directly.

Use names:

python main.py route.kmz --url-mode name

Use coordinates:

python main.py route.kmz --url-mode coord

No prompt is displayed.


CSV Export Options

The application supports multiple CSV export workflows.


Save Dialog

If no CSV file is supplied, a Save As dialog appears.

The user can:

  • Choose location
  • Change filename
  • Cancel export

If cancelled:

CSV export cancelled.

No file is created.


Specify CSV File

Example:

python main.py route.kmz --csv points.csv

or

python main.py route.kmz --csv "C:\Routes\points.csv"

CSV is written directly without prompting.


Disable CSV Export

Example:

python main.py route.kmz --no-csv

The application skips CSV generation entirely.


Fully Automated Example

This example runs without any dialogs or prompts.

python main.py route.kmz \
    --url-mode coord \
    --csv points.csv

Workflow:

  1. Load KMZ file
  2. Extract route points
  3. Generate coordinate-based Google Maps URL
  4. Save CSV file

Perfect for automation scripts and scheduled jobs.


Fully Interactive Example

python main.py

Workflow:

  1. Select KML/KMZ file
  2. Choose URL generation mode
  3. Select CSV save location
  4. Receive Google Maps URL

Ideal for occasional users.


Why Split the Project into a Library and CLI?

The original version mixed:

  • Parsing logic
  • URL generation
  • User interaction
  • File dialogs

inside a single script.

By separating the code:

Benefits for Developers

  • Easier testing
  • Better maintainability
  • Cleaner architecture
  • Reusable functionality
  • Simpler future enhancements

Benefits for Users

  • Interactive desktop experience
  • Command-line automation
  • Flexible workflows

Future Enhancements

Potential improvements include:

  • GPX file support
  • Route statistics
  • Distance calculations
  • Elevation analysis
  • OpenStreetMap URL generation
  • GeoJSON export
  • Interactive map visualization
  • Batch processing of multiple files

Conclusion

By separating the project into a reusable KmlRouteParser library and a lightweight command-line application, the solution becomes useful for both developers and end users.

Developers can import the library directly into their applications, while non-technical users can continue to use graphical file dialogs and interactive prompts.

The result is a flexible tool capable of parsing KML/KMZ files, generating Google Maps route URLs, exporting CSV data, and fitting seamlessly into both manual and automated workflows.

Posted in PythonTagged , , ,

How to Build a Docker SMTP Relay on Ubuntu Using Postfix

If your applications need to send emails reliably, an SMTP relay is one of the cleanest solutions.

In this tutorial, we will build a lightweight SMTP relay using Docker and Postfix on Ubuntu. Your applications will send email locally to the relay, and the relay will securely forward mail through providers like Amazon SES, SendGrid, Mailgun, or Gmail SMTP.

This setup is ideal for:

  • Laravel applications
  • WordPress websites
  • Node.js apps
  • Dockerized services
  • Internal notification systems
  • Transactional emails

Architecture

Application
    ↓ SMTP
Docker Postfix Relay
    ↓ TLS SMTP
Amazon SES / SendGrid / Mailgun
    ↓
Recipient Inbox

Prerequisites

Before starting, make sure you have:

  • Ubuntu server
  • Docker installed
  • Docker Compose plugin installed
  • SMTP provider credentials

Supported providers include:

  • Amazon SES
  • SendGrid
  • Mailgun
  • Gmail SMTP
  • Postmark

Step 1 — Install Docker

Update Ubuntu:

sudo apt update

Install Docker:

sudo apt install -y docker.io docker-compose-plugin

Enable Docker:

sudo systemctl enable --now docker

Verify installation:

docker --version

Optional: run Docker without sudo

sudo usermod -aG docker $USER
newgrp docker

Step 2 — Create Project Directory

Create a working directory:

sudo mkdir -p /opt/smtp-relay
cd /opt/smtp-relay

Step 3 — Create Persistent Storage

Create directories for mail queue and logs:

sudo mkdir -p relay
sudo mkdir -p logs

These directories ensure queued emails survive container restarts.


Step 4 — Create Docker Compose File

Create a docker-compose.yml file:


services:
  smtp-relay:
    image: boky/postfix
    container_name: smtp-relay
    restart: unless-stopped

    ports:
      - "25:25"

    environment:
      # Upstream SMTP provider
      RELAYHOST: smtp.gmail.com
      RELAYHOST_PORT: 587
      RELAYHOST_USERNAME: YourSMTPEnabledGmailUserID
      RELAYHOST_PASSWORD: YourGmailPassword

      # Allowed sender domains
      ALLOWED_SENDER_DOMAINS: wempro.com,pumpsandinstrumentations.com

      # Relay hostname
      POSTFIX_myhostname: relay.vmi3202307.local
      POSTFIX_mynetworks: 127.0.0.0/8 172.16.0.0/12 192.168.0.0/16

      POSTFIX_smtpd_recipient_restrictions: permit_mynetworks,reject_unauth_destination

      TZ: UTC

    volumes:
      # Mail queue persistence
      - ./relay:/var/spool/postfix

      # Optional logs
      - ./logs:/var/log

    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"



Save the file.


Step 5 — Start the SMTP Relay

Launch the container:

docker compose up -d

Verify container status:

docker ps

View logs:

docker logs -f smtp-relay

Step 6 — Test Email Sending

Install swaks:

sudo apt install -y swaks

Send a test email:

swaks \
  --to you@example.com \
  --from noreply@yourdomain.com \
  --server localhost:25 \
  --header "Subject: SMTP Relay Test" \
  --body "SMTP relay is working"

Successful output:

250 2.0.0 Ok: queued as ...

Step 7 — Configure Your Application

Your applications should connect to:

localhost:25

Example DSN:

smtp://localhost:25

Laravel .env example:

MAIL_MAILER=smtp
MAIL_HOST=localhost
MAIL_PORT=25
MAIL_USERNAME=null
MAIL_PASSWORD=null
MAIL_ENCRYPTION=null
MAIL_FROM_ADDRESS=noreply@yourdomain.com
MAIL_FROM_NAME="Your App"

Multiple Domain Support

To allow multiple sender domains:

ALLOWED_SENDER_DOMAINS: domain1.com,domain2.com,domain3.com

Why Use an SMTP Relay?

Benefits include:

  • centralized email handling
  • provider abstraction
  • email queueing
  • retry handling
  • cleaner application configuration
  • rate limiting
  • easier provider switching

Important Security Tips

Do NOT Create an Open Relay

Never use:

POSTFIX_mynetworks: 0.0.0.0/0

This will allow the internet to abuse your server for spam.


SPF, DKIM, and Deliverability

For production use, verify your domain with your SMTP provider and configure:

  • SPF
  • DKIM
  • DMARC

Without these, emails may land in spam folders.


Queue Management

View mail queue:

docker exec -it smtp-relay postqueue -p

Flush queue:

docker exec -it smtp-relay postqueue -f

Final Thoughts

A Dockerized SMTP relay is a lightweight and reliable solution for modern applications. By combining Postfix with providers like Amazon SES or SendGrid, you get:

  • reliable delivery
  • secure outbound SMTP
  • local application integration
  • retry and queue management
  • simplified infrastructure

This setup works especially well for Docker-based deployments and internal application stacks.

Happy emailing!

Posted in ubuntuTagged , , , , , ,

Testing MySQL Connectivity Using a Dockerized Python Container

When working with microservices, CI pipelines, or containerized environments, it’s often useful to verify MySQL connectivity independently of your application code.

In this post, we’ll build a lightweight Docker container that:

  • Connects to a MySQL database
  • Accepts all connection parameters at runtime
  • Waits and retries until MySQL is available
  • Fails fast if configuration is missing
  • Supports connection timeouts
  • Lists up to 5 tables if the user has visibility or proper privileges
  • Returns CI-friendly exit codes

This approach is ideal for smoke tests, health checks, and CI/CD pipelines.

Why a Dedicated MySQL Connection Tester?

Common scenarios where this is useful:

  • Verifying database access in CI before running migrations
  • Testing credentials in Kubernetes or Docker Compose
  • Debugging network or permission issues
  • Smoke-testing production or staging databases safely

Instead of baking logic into your app, we use a single-purpose container.

Solution Overview

We will build:

  • A Python-based Docker image
  • Runtime-configurable MySQL connection
  • Built-in retry and timeout logic
  • Optional table listing (non-fatal if permissions are missing)

Final Dockerfile

FROM python:3.11-slim

WORKDIR /app

RUN pip install mysql-connector-python

RUN echo "import os" > test_db.py && \
    echo "import sys" >> test_db.py && \
    echo "import time" >> test_db.py && \
    echo "import mysql.connector" >> test_db.py && \
    echo "" >> test_db.py && \
    echo "REQUIRED_VARS = ['MYSQL_HOST', 'MYSQL_USER', 'MYSQL_PASSWORD', 'MYSQL_DATABASE']" >> test_db.py && \
    echo "missing = [v for v in REQUIRED_VARS if not os.getenv(v)]" >> test_db.py && \
    echo "if missing:" >> test_db.py && \
    echo "    print('Missing required environment variables: ' + ', '.join(missing))" >> test_db.py && \
    echo "    sys.exit(1)" >> test_db.py && \
    echo "" >> test_db.py && \
    echo "MAX_RETRIES = int(os.getenv('MYSQL_MAX_RETRIES', 10))" >> test_db.py && \
    echo "RETRY_DELAY = int(os.getenv('MYSQL_RETRY_DELAY', 3))" >> test_db.py && \
    echo "CONNECT_TIMEOUT = int(os.getenv('MYSQL_CONNECT_TIMEOUT', 5))" >> test_db.py && \
    echo "" >> test_db.py && \
    echo "for attempt in range(1, MAX_RETRIES + 1):" >> test_db.py && \
    echo "    try:" >> test_db.py && \
    echo "        print(f'Attempt {attempt}: Connecting to MySQL...')" >> test_db.py && \
    echo "        conn = mysql.connector.connect(" >> test_db.py && \
    echo "            host=os.getenv('MYSQL_HOST')," >> test_db.py && \
    echo "            user=os.getenv('MYSQL_USER')," >> test_db.py && \
    echo "            password=os.getenv('MYSQL_PASSWORD')," >> test_db.py && \
    echo "            database=os.getenv('MYSQL_DATABASE')," >> test_db.py && \
    echo "            port=int(os.getenv('MYSQL_PORT', 3306))," >> test_db.py && \
    echo "            connection_timeout=CONNECT_TIMEOUT" >> test_db.py && \
    echo "        )" >> test_db.py && \
    echo "" >> test_db.py && \
    echo "        if conn.is_connected():" >> test_db.py && \
    echo "            print('MySQL connection successful!')" >> test_db.py && \
    echo "" >> test_db.py && \
    echo "            try:" >> test_db.py && \
    echo "                cursor = conn.cursor()" >> test_db.py && \
    echo "                try:" >> test_db.py && \
    echo "                    cursor.execute('SHOW TABLES')" >> test_db.py && \
    echo "                    tables = cursor.fetchmany(5)" >> test_db.py && \
    echo "                    source = 'SHOW TABLES'" >> test_db.py && \
    echo "                except mysql.connector.Error:" >> test_db.py && \
    echo "                    cursor.execute(" >> test_db.py && \
    echo "                        'SELECT table_name FROM information_schema.tables WHERE table_schema = %s LIMIT 5'," >> test_db.py && \
    echo "                        (os.getenv('MYSQL_DATABASE'),)" >> test_db.py && \
    echo "                    )" >> test_db.py && \
    echo "                    tables = cursor.fetchall()" >> test_db.py && \
    echo "                    source = 'information_schema.tables'" >> test_db.py && \
    echo "" >> test_db.py && \
    echo "                if tables:" >> test_db.py && \
    echo "                    print(f'Listing up to 5 tables using {source}:')" >> test_db.py && \
    echo "                    for t in tables:" >> test_db.py && \
    echo "                        print(' - ' + t[0])" >> test_db.py && \
    echo "                else:" >> test_db.py && \
    echo "                    print('Connected, but no tables found or insufficient privileges.')" >> test_db.py && \
    echo "" >> test_db.py && \
    echo "            except mysql.connector.Error as e:" >> test_db.py && \
    echo "                print('Connected, but cannot list tables: ' + str(e))" >> test_db.py && \
    echo "" >> test_db.py && \
    echo "            conn.close()" >> test_db.py && \
    echo "            sys.exit(0)" >> test_db.py && \
    echo "" >> test_db.py && \
    echo "    except Exception as e:" >> test_db.py && \
    echo "        print('Connection failed: ' + str(e))" >> test_db.py && \
    echo "        if attempt == MAX_RETRIES:" >> test_db.py && \
    echo "            print('Max retries reached. Exiting.')" >> test_db.py && \
    echo "            sys.exit(2)" >> test_db.py && \
    echo "        time.sleep(RETRY_DELAY)" >> test_db.py

CMD ["python", "test_db.py"]

Build the Image

docker build -t mysql-test-client .

Run the Container (Passing Parameters at Runtime)

sudo docker run --rm --network=esearchnetwork -e MYSQL_HOST=your_host -e MYSQL_PORT=port2connect -e MYSQL_USER=your_db -e MYSQL_PASSWORD=your_super_secret_password -e MYSQL_DATABASE=your_database -e MYSQL_CONNECT_TIMEOUT=10 -e MYSQL_MAX_RETRIES=5 -e MYSQL_RETRY_DELAY=2 mysql-test-client

Example Output

Successful connection with privileges

Attempt 1: Connecting to MySQL…
MySQL connection successful!
Listing up to 5 tables:
users
orders
products
invoices
logs

Successful connection without table privileges

MySQL connection successful!
Connected, but no tables found or insufficient privileges.

Exit Codes (CI-Friendly)

Exit CodeMeaning
0Connection successful
1Missing required environment variables
2MySQL unreachable after retries

Best Practices & Improvements

Recommended enhancements:

  • Replace echo with COPY test_db.py for maintainability
  • Output JSON for CI parsing
  • Add TLS / SSL parameters

Conclusion

This lightweight Docker-based MySQL connection tester is a reliable, reusable, and secure way to validate database access across environments.

It works equally well for:

  • Local development
  • Docker Compose
  • Kubernetes
  • CI/CD pipelines

If you need a cleaner version with a standalone Python file, or a Compose/Kubernetes variant, this setup is easy to extend.

Happy testing

Posted in mysql, PythonTagged , , ,

Jailed Ubuntu SFTP User

> Add a user as system user (which will prevent to create home directory) but without login capability

$ sudo adduser moderpatshala --system --shell /usr/sbin/nologin

>> That user need a password to login, you can skip it if you want to use public key authentication which is more secured than password login
$ sudo passwd moderpatshala
>> Now fix jail directory as root owned
$ sudo chown root:root /home/moderpatshala
>> Fix permission, chroot required relax file mode for root location like drwxr-xr-x
$ sudo chmod 755 /home/moderpatshala
>> Provide a writable directory under jailed directory for your sftp user
$ sudo chown -R moderpatshala /home/moderpatshala/public_html

>> Now you need to change SSH demon settings. You can add (if your sshd configuration settings allowed) a different file which I prefer
$ sudo vi /etc/ssh/sshd_config
————- or ———————-
$ sudo vi /etc/ssh/sshd_config.d/80-user-moderpatshala.conf

Match User moderpatshala
  PasswordAuthentication yes
  PubkeyAuthentication no
  ChrootDirectory /home/moderpatshala
  ForceCommand internal-sftp
  X11Forwarding no
  AllowTcpForwarding no

>> Finally it’s time to restart your sshd
$ sudo systemctl restart ssh

Posted in linux, ubuntuTagged , ,

Partition, format, mount and add shortcut to deck on Ubuntu

Reference:

  • https://www.howtogeek.com/106873/how-to-use-fdisk-to-manage-partitions-on-linux/
  • https://gist.github.com/keithmorris/b2aeec1ea947d4176a14c1c6a58bfc36
  • https://askubuntu.com/a/985273

List logical disks and partitions

sudo fdisk -l

Partition the disk

sudo fdisk /dev/sdb

  • Press n to create a partition
  • Press p or l to create primary or logical partitions
  • Press w to write your changes or q to quit

Format the partition

  • sudo mkfs -t ext4 /dev/sdb1
  • sudo mkfs -t ext4 -N 2000000 /dev/sdb1 – This will manually set the number of inodes to 2,000,000

Another Proven Format Command

sudo mkfs.ext4 /dev/sdb1

Mount disk

  • mount – Shows what is mounted
  • mkdir /mnt/mydrive
  • chown <your desired user>:<group> /mnt/mydrive
  • mount -t ext4 /dev/sdb1 /mnt/mydrive

Get disk’s UUID

ls -al /dev/disk/by-uuid/
or
blkid

Mount at boot

Add the following line to your /etc/fstab file adjusting the UUID to your device’s id and the directory to where you want to mount:

UUID=811d3de0-ca6b-4b61-9445-af2e306d9999 /mnt/mydrive ext4 defaults 0 0
mount -a - remounts filesystems from /etc/fstab

Add shortcut to Deck

Create a .desktop file (say custom-filemanager.desktop) in ~/.local/share/applications. You can do this by running the following command in Terminal

touch ~/.local/share/applications/custom-filemanager.desktop

Open the .desktop file using a text-editor, for example by running

gedit ~/.local/share/applications/custom-filemanager.desktop

Add the following lines to the file:

[Desktop Entry]
Name=File Manager
Comment=Access and organize files
Keywords=folder;manager;explore;disk;filesystem;
Exec=nautilus --new-window %U
Icon=org.gnome.Nautilus
Terminal=false
Type=Application
Categories=GNOME;GTK;Utility;Core;FileManager;
Actions=new-window;open-downloads;
[Desktop Action new-window]
Name=New Window
Exec=nautilus --new-window
[Desktop Action open-downloads]
Name=Open my Downloads folder
Exec=nautilus /home/YOUR-USER-NAME/Downloads
Replace YOUR-USER-NAME by your user-name in the last line.

Save the file.

Click on “Activities” and search for “File Manager”. It should appear.

Right click on the “File Manager” and select “Add to Favourites”. It should be added to the dock.

Posted in Uncategorized

Find Non Alpha numeric character in VS code

Now a days, I’m loving VS code for its resource management. I can focused on code and other required library for my project instead of IDE.

In my current project I have to migrate data from legacy system to new system and those data from legacy system has many special character which is non-alphanumeric character. I can create a script to do this job, but I was thinking why not I find a solution in VS code to find those character and take decision based on finding!!!

In VS code there has a search feature that support regular expression. So, I build a regular expression to find those characters. I’m writing this note to keep that regular expression on my record to use in future.

[^a-zA-Z();!0-~9\s,_'#-:\/[]?@"]

Hope this may help others (if they find my note) too.

Posted in ideTagged ,

SSL implementation for Django project in custom port

Here I’m going to implement SSL in Django project which will access through custom port. To do this I’m going to use Apache, for SSL I’m going to use letsencrypt and my Django project containerize in docker. I’m going to bring content through Apache proxy technique.

First of all we need to execute (any) Django project. Please follow Create Docker Container for Hello World with Django and uWsgi Server to create a simple Hello World Django project. We are going to uwsgi socket instead of http server. So, in Dockerfile last line need to change as follows –

ENTRYPOINT ["uwsgi", "--socket", ":9000", "--workers", "4", "--master", "--enable-threads", "--module", "helloworlddjango.wsgi"]

Now we have our project up and running. Now in Apache create entry for our domain and using certbot of letsencrypt install SSL. Please point document root in any safe location, we will use that document root to install SSL and then we will point our uWsgi server to bring content from our project. There has a lot of resource in internet to achieve this. Please configure such way that domain will redirect non-ssl to ssl url automatically.

After successfully access of domain securely we can move to change SSL port. To do this we need to change listen of Apache configuration. In my server I need to change /etc/apache2/ports.conf (it may vary server to server). Following commands need to use to access custom port –

<IfModule ssl_module>
	Listen 443
	Listen 59222
</IfModule>

Now we need to install “libapache2-mod-proxy-uwsgi” module to access content through Apache proxy technique.

sudo apt-get install libapache2-mod-proxy-uwsgi

Also we need to enable proxy at apache2

sudo a2enmod proxy
sudo a2enmod proxy_uwsgi

We are ready to access our Django project content. We need to append following configuration into our domain configuration of Apache server. In my case file location is /etc/apache2/sites-enabled/helpabodessltest.shahadathossain.com-le-ssl.conf

SSLProxyEngine on
SSLProxyVerify none 
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off
SSLProxyCheckPeerExpire off

ProxyRequests Off
ProxyPreserveHost On
ProxyPass / uwsgi://127.0.0.1:9000/ keepalive=On
ProxyPassReverse / uwsgi://127.0.0.1:9000/

Please note, we need to put these code inside “VirtualHost” block. Also need to change “VirtualHost” tag like as follows –

<VirtualHost *:59222>

Also we can put redirect code in Apache from http to https with custom port like following code. Note that we need to put this code into general (80) configuration of the domain (also inside VirtualHost block)

RewriteEngine on
RewriteCond %{SERVER_NAME} =helpabodessltest.shahadathossain.com
RewriteRule ^ https://%{SERVER_NAME}:59222%{REQUEST_URI} [END,NE,R=permanent]

That’s all, we need to restart Apache server. If everything goes fine we can visit our domain with custom port to see “Hello World” output in browser. Thanks.

Posted in apache, PythonTagged ,

Create Docker Container for Hello World with Django and uWsgi Server

I was searching a Hello World implementation for Django of Python in Docker container, but can’t find any good resource at online. So, I plan to code it myself and document it.

This is pure Docker implementation, you don’t need to create any project for Django. You just need Dockerfile to see “Hello World” at browser which powered by Django and uWsgi module.

Here is high level explanation that I’m going TODO –

  • Python, Pip and setuptools installation and upgrade
  • Create requirement.txt file
  • Execute requirement.txt file with Pip
  • Create Django project
  • Modify project settings to allow our domain in Django
  • Replace project’s urls.py to send “Hello World” string to output
  • Code to run server through uWsgi module

Entire steps I’ll do into a single Dockerfile, which we need to build and run through Docker. Here is step by step implementation of Dockerfile.

FROM python:3.11.3
WORKDIR /code
RUN pip install --upgrade pip
RUN pip install setuptools
RUN pip install -U setuptools

Its pretty straight forward, we are using Python 3.11.3 and install Pip and setuptools here.

RUN echo "Django==4.2" >> requirements.txt
RUN echo "uWSGI==2.0.25" >> requirements.txt
RUN pip install -r requirements.txt

Here we create requirement.txt file where we instruct to install Django version 4.2 and uWSGI module version 2.0.25 and then we execute the newly created requirements.txt through Pip.

RUN django-admin startproject helloworlddjango
WORKDIR /code/helloworlddjango
RUN echo "ALLOWED_HOSTS = ['127.0.0.1', 'localhost', 'helpabodessltest.shahadathossain.com']" >> helloworlddjango/settings.py

In this stage we created helloworld project with django-admin (we already Django installed) also we append our project’s settings.py to allow our domain. For this we just append “ALLOWED_HOSTS” variable value.

RUN echo "from django.urls import path" > helloworlddjango/urls.py
RUN echo "from django.shortcuts import HttpResponse" >> helloworlddjango/urls.py
RUN echo "def home_page_view_hello_world(request):" >> helloworlddjango/urls.py
RUN echo "    return HttpResponse('Hello World')" >> helloworlddjango/urls.py
RUN echo "urlpatterns = [path('', home_page_view_hello_world, name='helloworld'),]" >> helloworlddjango/urls.py

This part actually pure Python code we (re)writing our urls.py file where we actually put “Hello World” string when user visit home page of our project.

RUN adduser --disabled-password --no-create-home django
USER django
ENTRYPOINT ["uwsgi", "--http", ":9000", "--workers", "4", "--master", "--enable-threads", "--module", "helloworlddjango.wsgi"]

This is another part where we run our project through uwsgi module. We can run straightly by Django’s builtin server by “manage.py” but here I covered to run uwsgi server.

Here is link https://github.com/razonklnbd/django-hello-world-with-docker where you found complete Dockerfile

To build docker container you have to have docker in your system. After ensuring docker into system you can use following commands to build and run –

sudo docker build -t django-hello-world-mshk .
sudo docker run --name djangohelloworldmshk -d --network=host django-hello-world-mshk:latest

You need to execute into the location where you put your Dockerfile. Please feel free to change container tag and name. You may like following command of docker to see the log and to delete running container (in case you are debugging something)

sudo docker logs djangohelloworldmshk
sudo docker rm $(sudo docker stop $(sudo docker ps -a -q --filter ancestor=django-hello-world-mshk --format="{{.ID}}"))
sudo docker rmi django-hello-world-mshk

That’s all for today! Thanks.

Posted in linux, Python, webdevelopmentTagged , , , ,

Install secured Proftpd w/o database w/ virtual jailed user

Recently I need to install simple ftp server to provide access. I used Proftpd which is I believe is good (I used in small project). When I starting install, I faced some technical problem and overcome it. So, I think I should write my experience for my personal future reference.

  1. Install proftpd-basic (follow https://mtxserv.com/vps-server/doc/how-to-install-a-ftp-server-with-proftpd-debian-ubuntu or any other good document available by searching internet)
    1.a) Configure to use virtual user
    1.b) Add virtual user using “ftpasswd” command
  2. Configure jail option of proftpd configuration (read – https://portal.hostingcontroller.com/kb/a222/how-to-jail-ftp-users-using-proftpd-server.aspx)
    Remove # (uncomment) in front of below line
    DefaultRoot ~
  3. Configure passive ports
    3.a) At firewall allow 20, 21 and those passive ports (example below)
    ufw allow 49xxx:49999/tcp
    ufw reload
  4. Restart proftpd

-> Test ftp connection

Secure ftp connection with self-signed TLS:

  1. Follow TLS configuration part only from https://www.makeuseof.com/install-proftpd-on-ubuntu/ or any other good document available to configure TLS
  2. Replace “TLSProtocol” settings (follow https://serverfault.com/a/1023382)
    TLSProtocol TLSv1 TLSv1.1 TLSv1.2
  3. Restart proftpd

Now test using FTP client, you may see that host name different than server. As because we used self-signed this type of warning we can ignore.

Posted in linux, ubuntuTagged , , , ,

Apache Python3 Gunicorn

My journey to install Gunicorn to server Python project is not pleasant because of old Ubuntu system where Python version 3.5 installed but default Gunicorn not compatible with this version.

So, as suggested from gunicorn.org I need to install Gunicorn version 3 for Python 3 … The point is, I need to install this Gunicorn 3 at outside of my virtual environment.

First of all we need to change wsgi.py file in Python project in my case – “<project root>/helloworld/wsgi.py”

import os, sys
# add the hellodjango project path into the sys.path
sys.path.append('/home/django-helloworld/helloworld')

# add the virtualenv site-packages path to the sys.path
sys.path.append('/home/django-helloworld/myvenv/lib/python3.5/site-packages')

from django.core.wsgi import get_wsgi_application
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'helloworld.settings')
application = get_wsgi_application()

Above file I added two lines because while executing from gunicorn 3 (which installed outside of virtual environment i.e. into OS) Python can’t find Django or other project related package.

Posted in apache, linux, PythonTagged , , , ,