Python _thread Mutex

When Python programs create threads, each thread shares the same global memory as every other thread. Usually, but not always, multiple threads can safely read from shared resources without issue. Threads writing to shared resources are a different story because one thread could potentially overwrite the work of another thread.

This post demonstrates an example program shown in Programming Python: Powerful Object-Oriented Programming where threads acquire and release locks in the program. The locking mechanism ensures that only one thread has access to a shared resource at a time.

Code

Here is an example program with my own comments added.

import _thread as thread, time

# This mutex object is created by calling
# thread.allocate_lock()
# The mutex is responsible for synchronizing threads
mutex = thread.allocate_lock()


def counter(tid, count):
    for i in range(count):
        time.sleep(1)
        
        # The standard out is a shared resource
        # Unless the program controls access to the standard out
        # multiple threads can print to standard out at the same time
        # which results in garbage output
        
        # Acquire a lock
        mutex.acquire()
        
        # Now only the current thread can print to the console
        print('[{}] => {}'.format(tid, i))
        
        # Make sure to release the lock for other threads when finished
        mutex.release()


if __name__ == '__main__':
    for i in range(5):
        thread.start_new_thread(counter, (i, 5))

    time.sleep(6)
    print('Main thread exiting...')

Explanation

The program creates five threads, each of which needs access to the standard output stream. The standard output stream is a global object that all of the threads share, which means that each thread can call print at the same time. That isn’t ideal because we can get garbage output printed to the console if two threads call the print() statement at the same time.

The solution is to lock access to the standard output stream so that only one thread may use it at a time. We do this by creating a mutex object on line 6 in the program by using thread.allocate_lock(). When a thread needs a lock, it calls acquire() on the mutex. At that point, all other threads that need protected resources have to sit and wait for mutex.release().

It’s important to keep the operations between mutex.acquire() and mutex.release() as brief as possible. Only one thread can hold a lock at a time, so the longer one thread holds a lock, the longer other threads need to wait for their turn to use the lock. That naturally impacts the performance of the overall program.

References

Lutz, Mark. Programming Python. Beijing, OReilly, 2013.

Python Threading

The Python threading module provides an OOP solution to threading. The base class, threading.Thread, follows a Java like pattern to creating and joining threads. This post provides a threading demonstration with an example program found in Programming Python: Powerful Object-Oriented Programming.

Code

This is the example program with my own comments added.

import threading

# Thread is the base class for creating OOP Style Threads
# It has a run() method that contains the code that runs in a new thread
class MyThread(threading.Thread):
    def __init__(self, myId, count, mutex):
        self.myId = myId
        self.count = count
        self.mutex = mutex
        threading.Thread.__init__(self)

    # Everything inside of run is executed in a seperate thread
    def run(self):
        for i in range(self.count):
            with self.mutex:
                print('[{}] => {}'.format(self.myId, i))


if __name__ == '__main__':
    stdoutmutex = threading.Lock()
    threads = []
    for i in range(10):
        # Create the new Thread Object
        thread = MyThread(i, 100, stdoutmutex)
        
        # The thread doesn't actually start running until
        # start() is called
        thread.start()
        threads.append(thread)

    for thread in threads:
        # join() is used to synchronize threads
        # Calling join() on a thread makes the parent thread wait
        # until the child thread has finished
        thread.join()

    print('Main thread exiting...')

Explanation

The OOP approach to Python threads requires developers to extend the threading.Thread class. The Thread class provides high level methods that support threading such as start() and join() which we discuss shortly. It has also an empty run() method that developers need to override. All of the code placed in the run() method runs in a new thread.

We are still free to use locks with OOP threads. On line 20, we acquire a lock by calling threading.Lock(). The mutex is passed to the thread object’s constructor on line 24 and is used by the thread on line 15 to aquire a lock to that standard output stream.

It’s important to note that the new thread doesn’t actually run until we call start(). The start() method is what submits the thread to the thread pool so that the Python runtime can use the thread. Never call run() directly on thread object because doing so will keep the program single threaded. The run() method is called by the Python environment when it’s the thread’s turn to run.

The example program also uses the join() method. The join() method is used to make a parent thread wait until a child thread completes. The example program creates 10 threads and needs to wait until all of the threads are finished. This is done by entering a for-each loop on line 31 and then calling join() on each of the threads. When join() is called, the parent thread sleeps until child thread’s run() method is finished. When all 10 threads are finished, the program exits.

References

Lutz, Mark. Programming Python. Beijing, OReilly, 2013.

Python _thread Basic

Python 3 has the newer thread package, but the _thread package still exists for developers who are more comfortable with the 2.x API. This is a basic example derived from Programming Python: Powerful Object-Oriented Programming that demonstrates how to create threads using the _thread module.

Code

import _thread as thread, time


# This function will run in a new thread
def counter(tid, count):
    for i in range(count):
        # Simulate a delay
        time.sleep(1)
        # Print out the thread id (tid) and the current iteration
        # of our for loop
        print('[{}] => {}'.format(tid, i))


if __name__ == '__main__':
    # Enter a loop that creates 5 threads
    for i in range(5):
        # Start a new thread passing a callable and it's arguments
        # in the form of a tuple
        thread.start_new_thread(counter, (i, 5))

    time.sleep(6)
    print('Main thread exiting')

Explanation

This script creates five new threads using _thread.start_new_thread. Each time a new thread is created, the counter function is called and is passed a tuple of (i, 5). That tuple corresponds to the tid and count parameters of the counter function. Counter enters a loop that runs 5 times since 5 was passed to the second parameter of counter on line 19. It will print the thread id and current iteration of the loop.

Meanwhile, the for loop in the parent thread continues to iterate because thread.start_new_thread does not block the for loop in the main thread. By calling start_new_thread, the program’s execution runs both the for loop in the main thread and the counter function in parallel. Allowing programs to run multiple portions of code at the same time is what gives threads their power. For example, you may wish to use a thread to handle a long running database query while the user continues to interact with the program in the main thread.

One final note about threads in Python. Threads give the appearance of allowing programs to multitask and for all intents and purposes, that is what is happening in the program. Nevertheless, what is really happening is that the Python Virtual Machine is time slicing the computer instructions and allowing a few lines of code to run before switching to another set of instructions.

In other words, if a program has three threads, A, B, and C, then Thread A runs for a few moments, then Thread B, and finally Thread C. Note that there is no guarantee to the order in which threads run. It is possible that one thread may run more often than other threads or that the order of running threads is different each time.

References

Lutz, Mark. Programming Python. Beijing, OReilly, 2013.

Use Python to Launch Another Program

Many Python programs operate as wrappers to other programs. For example, there are a wide variety of Python programs that work as a GUI wrapper for CLI programs. This tutorial uses an example from Programming Python: Powerful Object-Oriented Programming to demonstrate how a Python program can be used to launch another program.

Code

Parent Process

This code represents a parent process that is capable of launching a child process. I added my own comments to help further explain the code.

import os

# Track the number of spawned children
spawned_children = 0

# Enter an infinite loop
while True:
    
    # Increment the number of children
    spawned_children += 1
    
    # Now launch a new process
    # If we are still in the parent process, pid is non-zero
    # otherwise, pid is zero and indicates that we are in the
    # child process
    pid = os.fork()
    
    # Test if we are in the child process
    if pid == 0:
        
        # Now, since we are in the new process, we use
        # os.execlp to launch a child script.
        os.execlp('python', 'python', 'child.py', str(spawned_children))
        assert False, 'error starting program'
    else:
        # If we are here, then we are still in the parent process
        print('Child is', pid)
        if input() == 'q': break

Child Process

This is a simple script that simply represents a sample child process.

import os, sys

# Parent passes the child process number
# as a command line argument
print('Hello from child', os.getpid(), sys.argv[1])

Explanation

We have two core concepts used in this program. The first concept is forking the parent process into a child process. The other concept is actually launching a new program.

Forking

Forking is the process of launching a copy of the running program. The program that launched a copy of itself is called the parent process, while the copy of the program is called the child process. In theory, a parent and its children can spawn as many children as the underlying operating system allows.

The example program spawns a child process on line 16 and assigns the result to the variable called pid. When os.fork() returns, pid is zero if we are running in the child process or non-zero if we are still in the parent. The program uses an if – else statement (lines 19 and 25 respectively) to perform different operations depending if we are in the parent or child process.

The parent process executes lines 27 and 28. It simply prints out the child’s process id and then asks the user if they wish to continue the program. The user can enter ‘q’ to exit the parent process.

The child process executes code on lines 23 and 24. Line 23 uses os.execlp to start the companion child script and passes the number of spawned children as a command line argument. The child script prints its pid and the number of child processes and then exits.

Launching Programs

The Python os module provides a variety of ways to launch other programs via a Python script. This example makes use of os.execlp which looks for an executable on the operating system’s path. The first argument is the name of the program followed by any number of command line arguments.

The os.execlp is not supposed to return to the caller because if successful, the program is replaced by the new program. This is why it’s necessary to call os.fork() prior to using os.execlp (unless you wish your program to simply end with the new process). When the example script calls os.execlp, the child process created by os.fork is replaced by the new program launched by os.execlp.

Line 24 handles the case when os.execlp fails. Should os.execlp fail, control is returned back to the caller. In most cases, a develop should notify the user that launching the new program has failed. The example script accomplishes this by using an assert False statement with an error message.

References

Lutz, Mark. Programming Python. Beijing, OReilly, 2013.

Python Basic Forking

Many programs need to execute tasks simultaneously and Python provides us with a few different mechanisms for concurrent programming. One of those mechanisms is called forking, where a call is made to the underlying operating system to create a working copy of a program that’s already running. The program that created the new process is called the parent process, while the processes that are created by the parent are called the child process.

This post shows the most basic form of creating processes in Python and helps serve as a foundation to understanding forking. The example is derived from Programming Python: Powerful Object-Oriented Programming, and I added my own comments to help better explain the program.

import os


# This is a function called by the child process
def child():
    # Use os.getpid() to get the pid for this process
    print('Hello from child', os.getpid())

    # force the child process to exit right away
    # or the child process will return to the infinite loop in parent()
    os._exit(0)


def parent():
    while True:
        # Attempt to fork this program into a new process. When forking is complete
        # newpid will be non-zero for the original process, but it wil be
        # 0 in the child process
        newpid = os.fork()

        # The program now goes in two different directions at the same time
        # When newpid is 0, we call child() and the child process exits
        if newpid == 0:  # Test if this is a child process
            child()
        else:
            # If are here, then we are still in the parent process
            # We print the pid of the parent and the child process (newpid)
            print('Hello from parent', os.getpid(), newpid)
        if input() == 'q':
            break


if __name__ == '__main__':
    parent()

When run on my machine, the program shows the following output

Hello from parent 87800 87802
Hello from child 87802
k
Hello from parent 87800 87803
Hello from child 87803
k
Hello from parent 87800 87804
Hello from child 87804
k
Hello from parent 87800 87805
Hello from child 87805
q

Explanation

The hardest part to grasp about this program is that when os.fork() is called on line 19, the program actually launches a copy of itself. The operating system creates the new process and that new process gets a copy of all variables in memory and execution of the new and old programs continue after line 19. (Note: The OS may not exactly copy the parent process, but functionally speaking, the child process can be considered to be a copy of the parent).

The os.fork() function returns a number called a PID (process ID). We can test the pid to see if we are running in the parent or child process. When we are in the child process, the value returned by os.fork() is zero. So on line 23, we test for 0 and if newpid is zero, we call the child() function.

The alternative case is that we are still running in the parent process (bearing in mind, that the child process is also running at this point in time as well). If we are still in the parent process os.fork() returns a non-zero value. In that case, we use the else block to print the parent and child PID.

The parent process continues to loop until the user enters q to quit. Each time the loop iterates, a new child process is created by the parent. The parent prints its own PID (using os.getpid()) and the pid of the child on line 28.

The child process also uses os.getpid() to get its own PID. It prints its own PID on line 7 and then on line 11, we use os._exit(0) to force the child process to shut down. This is a critical step for this program! If we were to omit the call to os._exit on line 11, the child process would return to the parent function and enter the same infinite loop the parent is using.

Conclusion

This is the most basic example of creating child processes using Python. Keep in mind that processes do not share memory (unlike threads). In real world programs, processes often need to sycnchronize data from one process to another process using tools such as network sockets, databases, or files. When a child process is spawned it gets a copy of the memory of the parent process, but then functions as an independent program.

Source

Lutz, Mark. Programming Python. Beijing, OReilly, 2013

Python os.walk

It’s very typical for a program to have to walk a file tree. In Recursion Example — Walking a file tree, I demonstrated how to use recursion to traverse a file system. Although it’s totally possible to walk through a file system in that fashion, it’s less than ideal because Python provides os.walk for this purpose.

The following script is a modified example borrowed from Programming Python: Powerful Object-Oriented Programming that demonstrates how to traverse a file system using os.walk.

import os
import sys


def lister(root):
    # os.walk returns a tuple with the current_folder, a list of sub_folders,
    # and a list of files in the current_folder
    for (current_folder, sub_folders, files) in os.walk(root):
        print('[' + current_folder + ']')
        for sub_folder in sub_folders:

            # Unix uses / as path separators, while Windows uses \
            # If we use os.path.join, we don't need to worry about which
            # path separator to use since os.path.join tracks that for us.
            path = os.path.join(current_folder, sub_folder)
            print('\t' + path)

        for file in files:
            path = os.path.join(current_folder, file)
            print('\t' + path)


if __name__ == '__main__':
    lister(sys.argv[1])

When run, this code prints out all of the files and directories starting at the specified root folder.

Explanation

os.walk

The os.walk function does the work of traversing a file system. The function generates a tuple with three fields. The first field is the current directory that os.walk is processing. The second field is a list of sub folders found in the current folder and the last field is a list of files found in the current folder.

Combining os.walk with a for loop is a very common technique (shown on line 8). The loop continues to iterate until os.walk finishes walking through the file system. The tuple declared in the for loop is updated on each iteration of the loop, providing developers with all of the information needed to process the contents of the directory.

os.path.join

Line 15 shows an example of using os.path.join to assemble a full path to a target folder or file. It’s import to use os.path.join to assemble file paths because Unix-like system use ‘/’ to separate file paths, while Windows systems use ‘\’. Tracking the path separator could be tedious work since it requires making a determination about which operating system is running the script. That’s not very ideal so Python provides os.path.join to take care of such work. As long as os.path.join is used, the assembled file paths will use the proper path separator for the os.

References

Lutz, Mark. Programming Python. Beijing, OReilly, 2013.

Python Line Scanner

This post borrows from a code example found in Programming Python: Powerful Object-Oriented Programming that demonstrates collecting command line arguments, opening a file, reading the file, and passing a function as a callback to another function.

Code

Here is the entire script that accepts a file as a command line argument and prints the contents of the file to the console.


def scanner(name, func):

    # Open the file (with statement ensures closure even if there is an exception)
    with open(name, 'r') as f:
        # Iterate through the file
        for line in f:
            # Call our callback function
            func(line)

if __name__ == '__main__':
    import sys
    name = sys.argv[1]

    # This is a function we are passing to scanner
    # Python has first class functions which can be
    # get passed as arguments to other functions
    def print_line(str):
        print(str, end='')

    # Call the scanner function, which in turn
    # calls the print_line function for each line
    # in the file
    scanner(name, print_line)

Command Line Arguments

The first concept covered in this script is processing command line arguments. Python requires us to import the sys module (line 12) which maintains an argv property. The argv property is a list-like object that contains all of the command line arguments used to hold all of the command line parameters. The first index [0] is the name of the script, followed by all of the other arguments supplied to the program.

On line 13, we grab the target file (stored in argv[1]) and keep it in a name variable. At this point, our program knows which file to the open later on when we use the scanner function.

First Class Functions

Python treats functions as objects. As such, we can define any function in a Python program and store it in a variable just like anything else. Lines 18-19 define a print_line function that accepts a String parameter. On line 24, print_line is the second argument to the scanner function.

Once inside of the scanner function, the print_line function is referenced by the variable func. On line 9, we call print_line with the func(line) rather than print_line(line). This works because func and print_line both refer to the same function object in memory. Passing functions in this fashion is incredibly powerful because it allows the scanner function to accept different behaviors for each line it processes.

For example, we could define a function the writes each line processed by scanner to a file rather than printing it to the console. Later on, we may choose to write another function that sends each line over the network via network sockets. The beauty of the scanner function as defined is that it works the same regardless of the callback function passed to the func argument. This programming technique is sometimes known as programming to a behavior.

Opening and Reading Files

The final topic covered is opening and reading a file. Line 5 in the script uses the with statement combined with the open function to actually open the file in read mode. The as f assigns the result of the open function to the variable f. The f variable holds a Python file object.

Since Python file objects support the iterator protocol, they can be used in for loops. On line 7, we read through each line in the file with the statement for line in f:. On each execution of the loop, the line variable is updated with the next line in the file.

When the loop is complete, the with statement calls the file’s close() method automatically, even if there is an exception. Of course, Python’s garabage collection will also ensure a file is closed, but this pattern provides an extra level of safety, especially since there are a variety of Python interpretors that may act differently than the CPython.

Conclusion

The most powerful take away from this example if the first class functions. Python treats functions like any other data type. This allows functions to be stored as passed around the program as required. Using first class functions keeps code loosely coupled and highly maintanable!

Sources

Lutz, Mark. Programming Python. Beijing, OReilly, 2013.

Python 3 Os File Tools

The Python os module has a number of useful file commands that allow developers to perform common file tasks such as changing file permissions, renaming files, or even deleting files. The following snippets are modified examples from Programming Python: Powerful Object-Oriented Programming

os.chmod

os.chmod alters a file’s permissions. The example usage below takes two arguments. The first argument is the path to the file and the second argument is a 9-bit string that composes the new file permissions.

os.chmod('belchers.txt', 0o777)

os.rename

os.rename is used to give a file a new name. The first argument is the current name of the file and the second argument is the new name of the file.

os.rename('belchers.txt', 'pestos.txt')

os.remove

The os.remove deletes a file. It takes the path of the target file to delete.

os.remove('pestos.txt')

os.path.isdir

The os.path.isdir accepts a path to a file or directory. It returns True if the path is a directory otherwise False.

os.path.isdir('/home') #True
os.path.isdir('belchers.txt') #False

os.path.isfile

os.path.isfile works like os.path.isdir but only it’s for files.

os.path.isfile('belchers.txt') #True
os.path.isfile('/home') #False

os.path.getsize

os.path.getsize returns the size of the file. It accepts the path to the file as an argument.

os.path.getsize('belchers.txt')

Sources

Lutz, Mark. Programming Python. Beijing, OReilly, 2013.

Kotlin Spring Data Delegation

Kotlin provides many features that can be really useful when working with Spring. I was doing a website for my fiancee where I found an excellent use case of Kotlin’s Delegation and Extension function that I am going to share with readers today.

Code

KotlinDelegationApplication.kt

package com.stonesoupprogramming.delegation.kotlindelegation

import org.hibernate.validator.constraints.NotBlank
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.boot.SpringApplication
import org.springframework.boot.autoconfigure.SpringBootApplication
import org.springframework.data.jpa.repository.JpaRepository
import org.springframework.stereotype.Controller
import org.springframework.stereotype.Service
import org.springframework.ui.Model
import org.springframework.validation.BindingResult
import org.springframework.web.bind.annotation.GetMapping
import org.springframework.web.bind.annotation.ModelAttribute
import org.springframework.web.bind.annotation.PostMapping
import org.springframework.web.bind.annotation.RequestMapping
import javax.persistence.Entity
import javax.persistence.GeneratedValue
import javax.persistence.Id
import javax.transaction.Transactional
import javax.validation.Valid
import javax.validation.constraints.NotNull

@SpringBootApplication
class KotlinDelegationApplication

enum class FamilyMemberType {Father, Mother, Daughter, Son}

//Basic entity class
@Entity
data class Belchers(
        @field: Id
        @field: GeneratedValue
        var id : Long? = null,

        @field: NotBlank(message = "Need a name!")
        var name : String = "",

        @field: NotNull(message = "Assign to a family type")
        var familyMemberType: FamilyMemberType? = null
)

//Now we are going to define a JpaRepository to handle persistence
interface BelchersRepository : JpaRepository

//Here is a service class that contains our business logic
@Service
@Transactional
class BelchersService(
        //Inject an instance of BelchersRepository
        @field : Autowired
        val belchersRepository: BelchersRepository) : BelchersRepository by belchersRepository {
    /**
     * The above line demonstrates Kotlin's delegation syntax. It works by specifying a variable whose type
     * is an interface (no concrete or abstract classes). After the colon, we specify the name of the interface
     * and the variable that provides the object we are using for delegation. The Kotlin compiler builds out all of
     * methods included in the interface and routes calls to those method to the delegate object.
     *
     * In this example, BelcherService gets all of the methods included in BelchersRepository and the belcherRepository
     * object handles the implementation of all BelcherRepository method unless we override them.
     */

    /**
     * Here is an example of where we override only one method of BelchersRepository
     *  so that we can customize the behavior.
     */
    override fun <s> save(entity: S): S {
        val formattedName = entity?.name?.split(" ")?.map { it.toLowerCase().capitalize() }?.joinToString(" ")
        if(formattedName != null){
            entity.name = formattedName
        }
        return belchersRepository.save(entity)
    }
}

//Example MVC controller
@Controller
@RequestMapping("/")
class IndexController (
        @field: Autowired
        val belchersService: BelchersService) {

    @ModelAttribute("belcherFamily")
    fun fetchFamily() = belchersService.findAll()

    @ModelAttribute("belcher")
    fun fetchBelcher() = Belchers()

    @GetMapping
    fun doGet() = "index"

    @PostMapping
    fun doPost(@Valid belcher : Belchers, bindingResult: BindingResult, model: Model) : String {
        var entity = belcher

        if(!bindingResult.hasErrors()){
            belchersService.save(belcher)
            entity = Belchers()
        }

        //Notice the use of extension functions to keep the code concise
        model.addBelcher(entity)
        model.addBelcherFamily()

        return "index"
    }

    //Some private extension functions which tend to be really useful in Spring MVC
    private fun Model.addBelcherFamily(){
        addAttribute("belcherFamily", belchersService.findAll())
    }

    private fun Model.addBelcher(belcher: Belchers = Belchers()){
        addAttribute("belcher", belcher)
    }
}

fun main(args: Array) {
    SpringApplication.run(KotlinDelegationApplication::class.java, *args)
}

index.html

<!DOCTYPE html>
<html lang="en" xmlns="http://www.w3.org/1999/xhtml"
      xmlns:th="http://www.thymeleaf.org">
<head>
    <meta charset="UTF-8">
    <title>Kotlin Delegation Example</title>

    <script src="http://code.jquery.com/jquery-3.2.1.js"
            integrity="sha256-DZAnKJ/6XZ9si04Hgrsxu/8s717jcIzLy3oi35EouyE="
            crossorigin="anonymous"></script>

    <!-- Latest compiled and minified CSS & JS -->
    <link rel="stylesheet" media="screen" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
    <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js"></script>

    <style>
        button {
            margin-top: 10px;
        }
    </style>
</head>
<body>
<div class="jumbotron">
    <div class="container">
        <h1>Kotlin Delegation</h1>
        <p>Web demonstration showing how Kotlin's delegation features pairs with Spring Data</p>
    </div>
</div>

<div class="container">
    <div class="row" th:if="${belcherFamily.size() > 0}">
        <div class="col-xs-12 col-sm-12 col-md-12 col-lg-12">
            <table class="table table-striped table-hover">
                <thead>
                <tr>
                    <th>ID</th>
                    <th>Name</th>
                    <th>Family Member Type</th>
                </tr>
                </thead>
                <tbody>
                <tr th:each="belcher : ${belcherFamily}">
                    <td th:text="${belcher.id}"></td>
                    <td th:text="${belcher.name}"></td>
                    <td th:text="${belcher.familyMemberType}"></td>
                </tr>
                </tbody>
            </table>
        </div>
    </div>

    <div class="row">
        <div class="col-xs-12 col-sm-12 col-md-12 col-lg-12">
            <form th:action="@{/}" method="post" th:object="${belcher}">
                <legend>Add a Family Member</legend>

                <div th:class="${#fields.hasErrors('name') ? 'form-group has-error' : 'form-group'}">
                    <label for="name">Name</label>
                    <input class="form-control" name="name" id="name" th:field="*{name}" />
                    <span th:if="${#fields.hasErrors('name')}" th:errors="*{name}" class="help-block"></span>
                </div>

                <select name="type" id="type" class="form-control" th:field="*{familyMemberType}">
                    <option th:each="value : ${T(com.stonesoupprogramming.delegation.kotlindelegation.FamilyMemberType).values()}"
                            th:value="${value}" th:text="${value}" />
                </select>

                <button class="btn btn-primary">Submit</button>
            </form>
        </div>
    </div>
</div>
</body>
</html>

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>

	<groupId>com.stonesoupprogramming.delegation</groupId>
	<artifactId>kotlin-delegation</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<packaging>jar</packaging>

	<name>kotlin-delegation</name>
	<description>Demo project for Spring Boot</description>

	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>1.5.6.RELEASE</version>
		<relativePath/> <!-- lookup parent from repository -->
	</parent>

	<properties>
		<kotlin.compiler.incremental>true</kotlin.compiler.incremental>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
		<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
		<java.version>1.8</java.version>
		<kotlin.version>1.1.3-2</kotlin.version>
		<thymeleaf.version>3.0.2.RELEASE</thymeleaf.version>
		<thymeleaf-layout-dialect.version>2.1.1</thymeleaf-layout-dialect.version>
	</properties>

	<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-data-jpa</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-thymeleaf</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-devtools</artifactId>
			<scope>runtime</scope>
		</dependency>
		<dependency>
			<groupId>org.hsqldb</groupId>
			<artifactId>hsqldb</artifactId>
			<scope>runtime</scope>
		</dependency>
		<dependency>
			<groupId>org.jetbrains.kotlin</groupId>
			<artifactId>kotlin-stdlib-jre8</artifactId>
			<version>${kotlin.version}</version>
		</dependency>
		<dependency>
			<groupId>org.jetbrains.kotlin</groupId>
			<artifactId>kotlin-reflect</artifactId>
			<version>${kotlin.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
		</dependency>
	</dependencies>

	<build>
		<sourceDirectory>${project.basedir}/src/main/kotlin</sourceDirectory>
		<testSourceDirectory>${project.basedir}/src/test/kotlin</testSourceDirectory>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>
			<plugin>
				<artifactId>kotlin-maven-plugin</artifactId>
				<groupId>org.jetbrains.kotlin</groupId>
				<version>${kotlin.version}</version>
				<configuration>
					<compilerPlugins>
						<plugin>spring</plugin>
					</compilerPlugins>
					<jvmTarget>1.8</jvmTarget>
				</configuration>
				<executions>
					<execution>
						<id>compile</id>
						<phase>compile</phase>
						<goals>
							<goal>compile</goal>
						</goals>
					</execution>
					<execution>
						<id>test-compile</id>
						<phase>test-compile</phase>
						<goals>
							<goal>test-compile</goal>
						</goals>
					</execution>
				</executions>
				<dependencies>
					<dependency>
						<groupId>org.jetbrains.kotlin</groupId>
						<artifactId>kotlin-maven-allopen</artifactId>
						<version>${kotlin.version}</version>
					</dependency>
				</dependencies>
			</plugin>
		</plugins>
	</build>

</project>

application.properties

spring.thymeleaf.mode= HTML
spring.thymeleaf.cache=false

Project Structure

structures copy

Explanation

Most developers are familiar with the delegation pattern. Delegation provides many of the same benefits as inheritence, but helps reduce issues such as fragile base classes or tight coupling to the base class. Kotlin’s delegation features go further by requiring developers to use an interface which helps promote loose coupling and programming to an interface. Since delegate objects aren’t part of an inheritance chain, we are free to use mutliple objects with delegation.

One of the huge drawbacks of using the delegation pattern in Java is the amount of work involved to use the pattern. Java requires developers to actually declare and implement each method of the delegate object. Although most IDE’s are happy to generate delegate methods, such methods require maintaince later on should an interface add or remove methods. This makes inheritence more attractive since the Java compiler adds or removes methods in child classes as they are added or removed in the base class without additional work from the developer.

The Kotlin compiler address the problems associated with developing delegate objects by generating the delegate methods for the developer. The Kotlin delegation syntax is found in KotlinDelegationApplication.kt on lines 48-51. As mentioned above, Kotlin requires the usage of interfaces when using delegation. This works nicely with Spring Data’s JPA template, since developers simply declare an interface that extends JpaRepository anyway. The delegation pattern is used in the BelchersService class, which takes an instance of BelchersRepository in its constructor and then uses the object to build out delegate methods.

At this point, BelcherService has the same methods as BelcherRepository without the need to generate boilerplate declarations and implementations to the delegate object. Since the code is loosely coupled, we are free to swap out different implementations of BelcherRepository as required. The code is easier to read because we are spared the boilerplate code required to implement the delegation pattern.

You may view the source at https://github.com/archer920/KotlinDelegation

Python Pipe Operations

Python programs can link their input and output streams together using pipe (‘|’) operations. Piping isn’t a feature of Python, but rather comes from operating system. Nevertheless, a Python program can leverage such a feature to create powerful operating system scripts that are highly portable across platforms. For example, developer toolchains can be scripted together using Python. I personally have used Python to feed input into unit testing for my Java/Kotlin programs.

This post is a modified example of a demonstration found in Programming Python: Powerful Object-Oriented Programming. It uses a producer and consumer script to demonstrate one program feeding input into another Python program.

writer.py

Here is the code for writer.py

family = [
    'Bob Belcher',
    'Linda Belcher',
    'Tina Belcher',
    'Gene Belcher',
    'Louise Belcher'
]
for f in family:
    print(f)

Nothing special here. We are just building up a list that prints out the names of our favorite TV family, the Belchers.

reader.py

This code receives the output from writer.

while True:
    try:
        print('Entering {}'.format(input()))
    except EOFError:
        break

Once again, this is a simple script. Without a pipe operation, the input() statement on line 3 would normally collect the input from the keyboard. That’s not what is going to happen here.

Demonstration

We are going to execute these scripts by running the command below in the terminal.

python writer.py | python reader.py

The pipe '|' character does the job of connecting writer.py's output stream to reader.py's input stream. Thus, print statements in writer.py connect to input statements in reader.py. Here is the output of this operation.

Entering Bob Belcher
Entering Linda Belcher
Entering Tina Belcher
Entering Gene Belcher
Entering Louise Belcher