Infinite Lists Python

Published 06 Jul 2014

When you need to solve a problem and use lists of unknown size, you should use generators to not waste memory. You can easily make use of generators to create infinite lists in Python.

def inf_list(start=0):
    x = start
    while True:
        x+=1
        yield x

#Never ending loop
for x in inf_list():
    print x

"prints 0, 1, 2 ..."


#More examples using custom operaotrs
from operator import add, sub

#Using add operator, increasing list
def inf_list(x=0, op=add):
    while True:
        yield x
        x = op(x, 1)

#Never ending loop..
for x in inf_list():
    print x

"prints 0, 1, 2 ..."


#Using sub operator, decreaseing list
def inf_list(x=0, op=sub):
    while True:
        yield x
        x = op(x, 1)

#Never ending loop..
for x in inf_list():
    print x

"prints 0, -1, -2..."

When and how to use this?

Say you want the first 1000 numbers divisible by 3, but you don’t know how large the list needs to be.

def inf_list(start=0):
    x = start
    while True:
        x+=1
        yield x

numbers_divisible_by_three = []

for x in inf_list():
    if len(numbers_divisible_by_three) >= 1000:
        break
    if(x%3)==0:
        numbers_divisible_by_three.append(x)

print numbers_divisible_by_three

More advanced usage

Generators are powerful and since they are iterators, we can create a more general solution to the previous example.

def inf_list(start=0):
    x = start
    while True:
        x+=1
        yield x

def filter_list(func, list, result_size=None):
    result = []
    for x in list:
        if result_size and len(result) >= result_size:
            break
        if func(x):
            result.append(x)
    return result

#Get the 1000 first elements divisible by given integer
print filter_list(lambda x: x % 1 == 0, inf_list(), 1000)
print filter_list(lambda x: x % 2 == 0, inf_list(), 1000)
print filter_list(lambda x: x % 3 == 0, inf_list(), 1000)
print filter_list(lambda x: x % 4 == 0, inf_list(), 1000)
print filter_list(lambda x: x % 150 == 0, inf_list(), 1000)

Use generators to calculate prime numbers

import itertools
import math


def inf_list(start=0):
    x = start
    while True:
        x += 1
        yield x


def filter_list(func, list, result_size=None):
    result = []
    for x in list:
        if result_size and len(result) >= result_size:
            break
        if func(x):
            result.append(x)
    return result


#Simple prime number number checker, just for this example
def is_prime_number(n):
    if n % 2 == 0 and n > 2:
        return False
    return all(n % i for i in range(3, int(math.sqrt(n)) + 1, 2))


#Get the first 1000 prime numbers
def get_n_first_prime_numbers(n):
    return filter_list(is_prime_number, inf_list(), n)


print get_n_first_prime_numbers(1000)


#Get the nth prime number
def get_nth_prime(n):
    return filter_list(is_prime_number, inf_list(), n + 1)[n]


print get_nth_prime(500)

Lazyness Python Generators

Published 05 Jul 2014

In Python we can iterate any iterable easy with a for in loop.

for x in list:
    do_something(x)

Usually we iterate through lists, for example [1,2,3]

for x in [1,2,3]:
    do_something(x)

We can generate new lists easily by using range(0,limit)

for x in range(0,10):
    do_something(x)

But what if the list is very large, what if you don’t really need the entire list.. It would be unnecessary to create the entire list up front. In Python we can use generators to make a partial list, xrange is a brilliant example of this. If we loop trough a xrange, instead for range, we don’t create the entire list up front, we only keep track of the “next” element in the list. The syntax is the same, the difference is that xrange will only evaluate the next element, so if we want to break the loop on a condition, we don’t waste memory.

for x in xrange(0,10):
    if(do_something(x)) == "done":
        break

But the real power comes when you create your own generators, it is very easy. This is a implementation of xrange.

def generator(start, stop):
    x = start
    while x<stop:
        x+=1
        yield x

for x in generator(0,10):
    if(do_something(x)) == "done":
        break

The fibonacci sequence as a generator function.

import itertools
import time

def fibonacci_sequence():
    a = 0
    b = 1

    while True:
        yield a
        a, b = a + b, a


#Convert the generator to a list, get last element, very slow!
def get_nth_fibonacci_number_sliced(n):
    return list(itertools.islice(fibonacci_sequence(), n))[-1]

t = time.time()
get_nth_fibonacci_number_sliced(500000)
print time.time() - t
"Output: 47.5731070042"

#Using enumerate, faster!
def get_nth_fibonacci_number(n):
    for index, fib in enumerate(fibonacci_sequence()):
        if index == n:
            return fib

t = time.time()
get_nth_fibonacci_number(500000)
print time.time() - t
"Output: 2.75447893143"

Mutation Rebinding Python

Published 01 Jul 2014

In Python, one of the most important and fundamental concepts are bindings vs mutations.

Let’s try one thing first, what happens if we do this?

l = [1, 2, 3, 4, 5, 6]

for x in l:
    x = 0

for i in l:
    print(i)

It is tempting to say we change the values in the list to zeroes. But why is that not happening? Whenver you say x = 0, or x = something, you rebind the x variable! It means to change what the x variable points to, so for every element in the list, we rebind x to 0, we don’t change the values in the list!

But is it possible to actually change the values in the list this way? Yes! Instead of rebinding the x variable, we can mutate the elements in the list. In order to mutate the value, we need an mutable objects, and int’s are not, we can create our own wrapper class called Integer, and mutate the value of each of the elements.

class Integer(object):
    def __init__(self, n):
        self.n = n

l = [Integer(1), Integer(2), Integer(3), Integer(4), Integer(5), Integer(6)]

for x in l:
    x.n = 0

for i in l:
    print i.n

In this example, we do not rebind the x variable, we rebind the x.n variable, and therefore change the value of x.n, whereas the x continue to be the element in the list. We mutate the x object. The result is a list of the same objects, containing differents values for the n variable.

Virtualbox Nat Ssh

Published 28 May 2013

From time to time, you need to set up a Virtualbox machine and connect with ssh. The problem is that the default network configuration use NAT, this mean the machine does not have it’s own IP-adress.

The easy solution is to set the machine in “Bridged mode”, which is shown in these screenshots. step1 step2

But in some network, this is not possible. Another solution is to create a reverse ssh tunnel. To do this you need to know your own machines ip address, and be able to log on to virtual machine in Virtualbox.

In the virtual machine terminal, type

$ ssh -R 4000:localhost:22 frecar@your-machine-ip

What this does is to open a reverse ssh connection from the virtual host to your computer. This means that any connection to port 4000 on your own machine wil redirect to port 22 on your virtual machine.

To connect to your virtual machine, type

$ ssh [email protected] -p 4000

Django Basis

Published 25 May 2013

django-basis is a simple reusable model for tracking time created, modified.

$ pip install django-basis

Nginx Gunicorn Django Ubuntu

Published 12 May 2013

First all of all, you have to set up a virtualenv. To do this, you need to have virtualenv installed

$ pip install virtualenv

Set up your environment, the –no-site-packages mean that this environment is totally isolated from your site-packages installed on your computer.

$ virtualenv venv --no-site-packages
$ cd venv
$ source activate