Category Archives: Programming

Singleton in Python

Singleton pattern should be the most known design pattern out there(maybe the most overused design pattern as well). The idea is to restrict the object creation to only one instance. The typical way of achieving this is to declare the Singleton class’s constructor as private and then provide a static method to do the object initiation. This method would create an object on it’s first call and then onward return the already created object. Since Python doesn’t have private constructors we have to go down a complete different path to achieve the same objective. Remember we have to somehow get between the object creation mechanism and make sure only one object is created at most. The right place to do this is __new__ which is where objects are created in Python. You can find a good introduction to __new__ here[1].

Here is an example Singleton class. What it does is check whether an object instance is already created and return it. If an object is not created yet it would call the super class’s __new__ where the object would be created and returned.


 class Singleton(object):
     __instance = None
     def __new__(cls, *args, **kwargs):
         if not cls.__instance:
             cls.__instance = super(Singleton, cls).__new__(cls,
*args, **kwargs)
         return cls.__instance
 

The downside of the above design is there is no simple way to subclass a singleton class.

Another popular pattern to achieve the same effect as the singleton pattern is the Borg pattern. This allows multiple object creation but forces each object to share the same state.

Here is an example Borg pattern.

class Borg(object):
    __state = {}
    def __init__(self):
        self.__dict__ = self.__state

[1] http://agiliq.com/blog/2012/06/__new__-python/

Advertisements

Preserving Remote IP/Host while proxying

You host your web application with a hosting provider. Your application log/access IP address of your users and you get 127.0.0.1 or some other private IP. Deja Vu? Most likely reason for the above scenario is your hosting provider is using a proxy and the proxy server sits in the same machine or in the same network. Under a such situation we end up using X-Forwarded-For header.

I am aware of two solutions to be used in such a proxy setup so the developer doesn’t have to end up using X-Forwarded-For header.

1.) When the proxy server is an Apache, using ProxyPreserveHost directive in mod_proxy.
This can be used to preserve the remote host not the remote ip. This is useful for situations where name based virtual hosting is used and the backend server needs to know the virtual name of host.
Open mod_proxy configuration file of your proxy server and enter directive, ProxyPreserveHost On, and restart your apache instance.

2.) When backend server is apache, use mod_rpaf
This apache module can be used to preserve both remote IP/HOST. Internally it uses X-Forwarded-For header to detect a proxy in it’s list of known proxies and reset the headers accordingly. This works with any proxy server in the front end provided that the proxy server sets X-Forwarded-For header. To use mod_rpaf, install and enable it in the backend server and add following directives in the module’s configuration.
RPAFenable On
RPAFsethostname On
RPAFproxy_ips 127.0.0.1

Remote IP is automatically preserved when RPAFenable On directive is used. RPAFsethostname On directive should be used to preserve host and RPAFproxy_ips is the list of known proxy ips.

Restart backend apache server and you are good to go.

Git over http(s)

Traditionally git used to work only over ssh or git protocols while there was only a dumb version of git over http which was slow and inefficient. While this was ok for most of the time sometimes git needs to be able to work over http. Now starting from git 1.7 both git servers and clients have support for smart http which works over http(s) and is supposed to be as efficient as the ssh version.

This functionality is made available by a cgi script called git-http-backend provided with git-core. So for git to work over http(s) there should be a web server already configured and as a result there won’t be any conflicts by both the web server and git trying acquire port 80.

The manual for the git-http-backend can be found here.

The following steps can be used to configure git to work over http(s) with Apache.

1) First configure Apache

Make sure mod_cgi, mod_alias, and mod_env are enabled.

Open the Apache config file and append the following. Debian based system should have it under /etc/apache2/apache2.conf by default

SetEnv GIT_PROJECT_ROOT /home/user/git_pub
SetEnv GIT_HTTP_EXPORT_ALL
ScriptAlias /git/ /usr/lib/git-core/git-http-backend/

The GIT_PROJECT_ROOT should point to the root folder where git repositories would be hosted. Set this away from the document root of the web server. What the above do is direct any requests with /git/ to the git-http-backend and tell the script that the root of git repositories is GIT_PROJECT_ROOT.

That is all that needs to be done that is specific to git over http(s). The manual for for the git-http-backend explains these steps pretty thoroughly.

Now for some tit-bits that are not explained in the manual. Those who are experienced with Apache and Git would find the following very boring.

2) For authentication for both read and write accesses append the following to theApache config file

<Location /git>
AuthType Basic
AuthName “Private Git Access”
AuthUserFile /etc/apache2/authusers
Require valid-user
</Location>

What the above do is make requests to /git only accessible to valid users and tell valid users are listed on the file /etc/apache2/authusers. Make sure the file authusers is accessible by Apache.

If there is no AuthUserFile in your system the following command can be used to create the user list at /etc/apache2/authusers and add the user ‘username’ to it. The command will prompt for a password for the user.

htpasswd -c /etc/apache2/authusers username

3) Restart Apache

On debian most probably, sudo /etc/init.d/apache2 restart

4) Create an empy bare git repository under the specified GIT_PROJECT_ROOT (/home/user/git_pub in our example)

cd to GIT_PROJECT_ROOT

mkdir project

cd project

git init –bare

5) Make the folder ‘project’ and it’s children owned by the user which the web server is run from. This should be done for push requests by clients to work or otherwise the web server won’t be able to merge files. On debian based systems this user is usually www-data and is defined in a file called envvars under apache2 installation.

sudo chown -R www-data project/
sudo chgrp -R www-data project/

Now the bare git repository should be pull-able and pushable by authorized users.

6) Clone the git repository over http(s) from another location

git clone http://username@host/git/project

7) Do the first commit

cd project
touch readme
git add readme
git commit -m “first commit”
git push origin master

It is as easy as that. From here the setuped git repository should work as normal.

Django cronjobs

In web applications sometimes you want to run processes that are seperated from the Request/Response cycle, like scheduled jobs that run at a specific time of a day. Scheduling such jobs in a unix environment is very easy with cronjobs. You just need to edit the crontab file and define a time and a command(a script to be run in this case) to execute at that time.

This post is about how to schedule such Django programs, that is python programs that import and use the Django framework.

This should be trivial because a Django program is another python program. But there is an important step to be done in about for Django modules to be discovered and run. That is to specify the Django settings file.

Let’s say there is a Django project called mysite and a Django app called myapp that has a method called runJob() in a python module called jobs.py that needs to be called every 30 mins.

let’s write a python script called thescript.py that will run every 30 minutes as a cronjob that will in turn call the runjob method.

thescript.py
#######################################

import os

os.environ[‘DJANGO_SETTINGS_MODULE’] = ‘settings’

from myapp.jobs import runJob

runJob()

###################################

For easiness save the ‘thescript.py’ in the mysite folder.

Then open a console and enter crontab -e to edit your cron file. Enter the following line.

*/30 * * * * python /path to mysite/thescript.py > logfile 2>&1

This will run the ‘thescript.py’ file every 30 minutes and any output would be directed to the logfile specified and the last ‘2>&1’ tells that no email should be sent to the administrator. For more information on how to write the crontab file have a look at this.

That is how easy it is to schedule a Django cronjob in an Unix environment.

Django-resolvetime

Django-resolvetime is a Django filter to convert a python datetime object into a format that specifies the time difference between the time of the datetime and now. I developed this for a personal project of mine and decided to opensource since I could not find a filter available to get the job done.

The code is hosted at http://code.google.com/p/django-resolvetime/. Installation and usage of the filter is straight forward. Instructions are given in the README file.

Configuring PyDev to work with Django

Django is the most widely used if not the most popular web framework among those available for python. Have a look at this report by Jacob Kaplan-Moss the co-creator of Django, to get an idea about the current standing of Django among the community.

One disadvantage for Django is the lack of IDE support it has. Here it holds a clear disadvantage against it’s rival web frameworks such as Ruby on Rails. Wing IDE looks impressive but is not free. I wasn’t much impressed with Komodo and it is not free either. The best bet seams to be Netbeans 7 which promises to have good python support. Hopefully it would have satisfactory Django support as well. Currently netbeans is having a python plugin that is still in beta stage. The best alternative for me as of now is PyDev which I have been using with python for sometime now. Here I will explain how to configure PyDev to work with Django.

1.) First download and install Django
2.) Add Django source folder to PYTHONPATH in pydev

  • Go to windows > preferences
  • In the opended dialog choose interpreter-python
  • Here under python interpreter a python interpreter should be available. If not add one by clicking new
  • In System PYTHONPATH add the downloaded source folder of Django by clicking New Folder

3.) Create a new pydev project.

  • Go to file > new > pydev project
  • Enter project name, path and importantly make sure the option ‘create default src folder and add it to PYTHONPATH?’ is choosen. This is easiest way or you can manually add the source folder of the new project to PYTHONPATH by going to File > properties > PYTHONPATH

4.) Start a new Django project

  • Open a new console from the Operating System(console support inside pydev is avialable but is not very satisfactory)
  • Navigate to the newly create src folder inside the pydev project
  • type django-admin.py startproject ‘sitename’ (sitename should be the name of your project). This how a new project is started in Django

If your projects name is mysite the folder structure should be as follows
mysite
src
mysite

Now pydev should be configured to work with Django. All the python feature that pydev supports should be now working with Django as well.

The standard features of a modern IDE works beautifully with pydev for Django but there are some major drawbacks. Django template tags are not supported in pydev. Also the console support inside pydev is not impressive. So for now I am forced to use pydev for Django coding, the OS console for giving Django commands and nothing for Django templates.

University of Moratuwa tops GSOC 2009

University of Moratuwa has been able to top the Google Summer of Code program, yet again. With 22 participating students university of Moratuwa was far ahead from the second place, University of Campinas Brazil which had 12 participating students. I’m glad i was a part of this success. Here are some stats of GSOC 2009.