the fact of real shit

Create Docker Container for Hello World with Django and uWsgi Server

I was searching a Hello World implementation for Django of Python in Docker container, but can’t find any good resource at online. So, I plan to code it myself and document it.

This is pure Docker implementation, you don’t need to create any project for Django. You just need Dockerfile to see “Hello World” at browser which powered by Django and uWsgi module.

Here is high level explanation that I’m going TODO –

  • Python, Pip and setuptools installation and upgrade
  • Create requirement.txt file
  • Execute requirement.txt file with Pip
  • Create Django project
  • Modify project settings to allow our domain in Django
  • Replace project’s urls.py to send “Hello World” string to output
  • Code to run server through uWsgi module

Entire steps I’ll do into a single Dockerfile, which we need to build and run through Docker. Here is step by step implementation of Dockerfile.

FROM python:3.11.3
WORKDIR /code
RUN pip install --upgrade pip
RUN pip install setuptools
RUN pip install -U setuptools

Its pretty straight forward, we are using Python 3.11.3 and install Pip and setuptools here.

RUN echo "Django==4.2" >> requirements.txt
RUN echo "uWSGI==2.0.25" >> requirements.txt
RUN pip install -r requirements.txt

Here we create requirement.txt file where we instruct to install Django version 4.2 and uWSGI module version 2.0.25 and then we execute the newly created requirements.txt through Pip.

RUN django-admin startproject helloworlddjango
WORKDIR /code/helloworlddjango
RUN echo "ALLOWED_HOSTS = ['127.0.0.1', 'localhost', 'helpabodessltest.shahadathossain.com']" >> helloworlddjango/settings.py

In this stage we created helloworld project with django-admin (we already Django installed) also we append our project’s settings.py to allow our domain. For this we just append “ALLOWED_HOSTS” variable value.

RUN echo "from django.urls import path" > helloworlddjango/urls.py
RUN echo "from django.shortcuts import HttpResponse" >> helloworlddjango/urls.py
RUN echo "def home_page_view_hello_world(request):" >> helloworlddjango/urls.py
RUN echo "    return HttpResponse('Hello World')" >> helloworlddjango/urls.py
RUN echo "urlpatterns = [path('', home_page_view_hello_world, name='helloworld'),]" >> helloworlddjango/urls.py

This part actually pure Python code we (re)writing our urls.py file where we actually put “Hello World” string when user visit home page of our project.

RUN adduser --disabled-password --no-create-home django
USER django
ENTRYPOINT ["uwsgi", "--http", ":9000", "--workers", "4", "--master", "--enable-threads", "--module", "helloworlddjango.wsgi"]

This is another part where we run our project through uwsgi module. We can run straightly by Django’s builtin server by “manage.py” but here I covered to run uwsgi server.

Here is link https://github.com/razonklnbd/django-hello-world-with-docker where you found complete Dockerfile

To build docker container you have to have docker in your system. After ensuring docker into system you can use following commands to build and run –

sudo docker build -t django-hello-world-mshk .
sudo docker run --name djangohelloworldmshk -d --network=host django-hello-world-mshk:latest

You need to execute into the location where you put your Dockerfile. Please feel free to change container tag and name. You may like following command of docker to see the log and to delete running container (in case you are debugging something)

sudo docker logs djangohelloworldmshk
sudo docker rm $(sudo docker stop $(sudo docker ps -a -q --filter ancestor=django-hello-world-mshk --format="{{.ID}}"))
sudo docker rmi django-hello-world-mshk

That’s all for today! Thanks.

Posted in linux, Python, webdevelopmentTagged , , , ,

Install secured Proftpd w/o database w/ virtual jailed user

Recently I need to install simple ftp server to provide access. I used Proftpd which is I believe is good (I used in small project). When I starting install, I faced some technical problem and overcome it. So, I think I should write my experience for my personal future reference.

  1. Install proftpd-basic (follow https://mtxserv.com/vps-server/doc/how-to-install-a-ftp-server-with-proftpd-debian-ubuntu or any other good document available by searching internet)
    1.a) Configure to use virtual user
    1.b) Add virtual user using “ftpasswd” command
  2. Configure jail option of proftpd configuration (read – https://portal.hostingcontroller.com/kb/a222/how-to-jail-ftp-users-using-proftpd-server.aspx)
    Remove # (uncomment) in front of below line
    DefaultRoot ~
  3. Configure passive ports
    3.a) At firewall allow 20, 21 and those passive ports (example below)
    ufw allow 49xxx:49999/tcp
    ufw reload
  4. Restart proftpd

-> Test ftp connection

Secure ftp connection with self-signed TLS:

  1. Follow TLS configuration part only from https://www.makeuseof.com/install-proftpd-on-ubuntu/ or any other good document available to configure TLS
  2. Replace “TLSProtocol” settings (follow https://serverfault.com/a/1023382)
    TLSProtocol TLSv1 TLSv1.1 TLSv1.2
  3. Restart proftpd

Now test using FTP client, you may see that host name different than server. As because we used self-signed this type of warning we can ignore.

Posted in linux, ubuntuTagged , , , ,

Apache Python3 Gunicorn

My journey to install Gunicorn to server Python project is not pleasant because of old Ubuntu system where Python version 3.5 installed but default Gunicorn not compatible with this version.

So, as suggested from gunicorn.org I need to install Gunicorn version 3 for Python 3 … The point is, I need to install this Gunicorn 3 at outside of my virtual environment.

First of all we need to change wsgi.py file in Python project in my case – “<project root>/helloworld/wsgi.py”

import os, sys
# add the hellodjango project path into the sys.path
sys.path.append('/home/django-helloworld/helloworld')

# add the virtualenv site-packages path to the sys.path
sys.path.append('/home/django-helloworld/myvenv/lib/python3.5/site-packages')

from django.core.wsgi import get_wsgi_application
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'helloworld.settings')
application = get_wsgi_application()

Above file I added two lines because while executing from gunicorn 3 (which installed outside of virtual environment i.e. into OS) Python can’t find Django or other project related package.

Posted in apache, linux, PythonTagged , , , ,

Execute React JS in ntfs partition

I do development in mounted hard drive which is different than OS partition, also I like to use that mounted drive within different OS like windows and linux simultaneously. Which allow me portability of my code backup in different system.

Challenge is, I have to use such file system for that mounted drive which can accessible in most of the OS like windows, linux or iOS. And in this case NTFS is the best file system which is really portable. Problem for this FS is, it can’t support executable bit of linux which lead to raise many problem in React development. Like if you put any React project into that mounted drive, you can’t execute that code.

To solve this problem I take Docker as a solution. Simply, I create docker image of React project and execute that image. Here is the simple Dockerfile for React project –

FROM node:18
WORKDIR /app
COPY package.json ./
RUN npm install
RUN npm install -g npm@9.7.2
COPY . ./
EXPOSE 3000
CMD ["npm", "start"]

You just need to build this docker image and need to run. You can see docker log for any output for your React project.

Posted in ubuntu, windowsTagged , , , ,

Copy large amount of file to remote server using nohup tar and ssh

This command will copy large amount of file to remote server by compressing and decompressing on the fly. It saves time and bandwidth. It will execute in background, so you can detach current login session.

nohup sh -c “tar -c /any/directory/at/source/server/ | gzip -2 | ssh server-alias ‘cat | tar xz -C /target/directory/of/target/server/'” > /dev/null 2>&1 &

here nohup output sent to /dev/null that means i don’t want any nohup output. you can adjust its behavior.

I use the command for millions of file that occupied more than 500 GB.

Posted in linuxTagged , , ,

add new hard disk into ubuntu more than 2TB size

lsblk

will list available device

parted /dev/sdd

considered device “sdd” from lsblk output

(parted) mklabel gpt
(parted) mkpart primary ext3 0 100%
(parted) print
(parted) quit

mkfs.ext3 /dev/sdd1

this command will format this hard disk into ext3 file system as we instruct by parted command

mkdir /home/data

mounting point directory creating

mount -t ext3 /dev/sdd1 /home/data

mounting formatted hard disk into target directory

blkid

try to find ID of new hard disk to write into fstab so that after restart our hard disk will mount automatically

sample output of blkid

/dev/sdd1: UUID="20e4b16b-4d4c-4053-b6f4-a2c103f2db2f" TYPE="ext3" PARTLABEL="primary" PARTUUID="33658d68-726f-4398-992f-2aaafebe17ff"

vi /etc/fstab

give an entry for our new hard disk at the end of this file

example entry

UUID=20e4b16b-4d4c-4053-b6f4-a2c103f2db2f /home/data ext3 nofail 0 0
Posted in linux, ubuntuTagged , , , ,

recursive change group and file mode in linux and detach active login session

change group

nohup sh -c “find /any/path/that/need/to/change/* -group mygroup -exec chgrp www-data {} \;” > /dev/null 2>&1 &

traditional way

nohup sh -c “chgrp -R www-data /any/path/that/need/to/change” > /dev/null 2>&1 &

change mode

nohup sh -c “find /any/path/that/need/to/change/* -perm u=rw,g=r,o=r -execdir chmod g+w {} \;” > /dev/null 2>&1 &

traditional way

nohup sh -c “chmod -R g+w /any/path/that/need/to/change” > /dev/null 2>&1 &

Posted in linux, ubuntuTagged , , , , , ,

Install php 7.2 ssh2 in ubuntu 16x

recently i need to install ssh2 connection through my php code to connect remote server by sftp for file transfer. i’m using php 7.2 and face a “sigment fault” issue for normal installation. i solved this issue by install required demon. hope it may help someone who face same issue.

first you need to install/upgrade some basic program

apt-get install gcc make autoconf libc-dev pkg-config

then install base library

apt-get install libssh2-1-dev

now install required php modules

apt-get install php7.2-dev php-pear

now install ssh by pecl (the most important part of installation)

— pecl channel-update pecl.php.net
pear config-show
— pear config-set php_ini /etc/php/7.2/apache2/php.ini
— pear config-set temp_dir /etc/php/temp/pear
pecl install ssh2-1.1.2

ignore commented line OR use if you understand by yourself

now you need to enable ssh2 extension into your php cli installation

echo “extension=ssh2.so” > /etc/php/7.2/mods-available/ssh2.ini
ln -s /etc/php/7.2/mods-available/ssh2.ini /etc/php/7.2/cli/conf.d/30-ssh2.ini

please check you php installation/configuration path. set priority on your own. i set here 30 without proper understanding 😀

now another important part is your PHP code. when we use ssh2 in fopen wraper, in other version of ssh2 connection we need to open a connection and we can use resource id. but with the above change you must use user, password, port i.e. full access information every time we need to connect to server. here is my sample code –

$fh = @fopen(‘ssh2.sftp://’ . $this->user.’:’.$this->pass.’@’.$this->ip.’:’.(intval($this->port)>0?intval($this->port):22) . $pRemoteLocation, $pMode);

all other code like directory creation or any other command execution may be same as before.

Posted in php, ubuntuTagged , , , , , , ,

change mysql data directory ubuntu 17.xx

Follow https://www.digitalocean.com/community/tutorials/how-to-move-a-mysql-data-directory-to-a-new-location-on-ubuntu-16-04
NOTE: you can skip empty directory (/var/lib/mysql/mysql) creation script in ubuntu 17.xx version.

Then try to start mysql server. If fail then log messages are telling you that /usr/sbin/mysqld needs read (r) access to open /proc/14767/status, /sys/devices/system/node/ (trailing slash because it wants to read the directory), and /proc/14767/task/14767/mem. The file to edit is /etc/apparmor.d/usr.sbin.mysqld.

In my case I solved the problem by adding these lines somewhere in the middle (with two spaces in front of each):

/proc/*/status r,
/sys/devices/system/node/ r,
/sys/devices/system/node/** r,

Reload apparmor:

sudo apparmor_parser -r /etc/apparmor.d/usr.sbin.mysqld
sudo /etc/init.d/apparmor reload

After doing that, try starting MySQL, and if you get more errors, add those files too and try again.

In some case mysql may fail to start for assoc i/o error. So, you may need to disable this settings by adding following line into /etc/mysql/mysql.conf.d/mysqld.cnf

innodb_use_native_aio=0

Another note about file permission of data directory is, “mysql” user should have permission into every directory of that path. Like if new data directory is “/media/myusb01/mysql-data” then “mysql” user should read access of “/media” folder and “/media/myusb01” folder and read write access at new data directory.

Hope mysql will start now.

Posted in linux, mysql, ubuntu

network adapter at guest [ubuntu] while setup host only network at virtualbox

I’m trying to setup host only network at virtualbox by following http://christophermaier.name/2010/09/01/host-only-networking-with-virtualbox/

but facing to choice network adapter at guest [ubuntu]

To see a list of the adapters your virtual machine can access, use this:

ls /sys/class/net

This will show you the interface names. For example, with two network adapters set up, my Ubuntu guest looks like this.

eth1 eth2 lo

To see a list of active network with IP address assigned

ifconfig

So you can easily guess what network adapter should use for host only network.

Well in a brand new ubuntu machine the permanent network save is not work in the way described in that article. For this you need to install ifupdown or ifupdown2 package

apt install ifupdown2

After install execute following command

ifup eth1

Then reboot.

Posted in linux, ubuntu, virtualbox