Kotlin Serializable Classes

We are free to write our own serializable classes. All we need to do is have our class implement the java.io.Serializable interface to mark it as a candidate for serialization. All non-transient fields are written to the output stream and restored by the input stream. Here is a program that demonstrates using serialization with our own classes.

import java.io.*

data class Cook(val name : String = "Bob",
                val job : String = "Cook",
                @Transient val age : Int = 44) : Serializable

fun main(args : Array<String>){
    val file = "bob.ser"

    val tiredBob = Cook()
    println("Before Serialization")
    println(tiredBob)

    ObjectOutputStream(FileOutputStream(file)).use{ it -> it.writeObject(tiredBob)}

    println("Bob has been serialized")
    println()
    println("Time to wake Bob up")

    ObjectInputStream(FileInputStream(file)).use { it ->
        val restedBob = it.readObject()

        when (restedBob){
            is Cook -> {
                println("Does Bob remember his age?")
                println(restedBob)
            }
            else -> println("Failed to restore Bob")
        }
    }
}

Our program defines a Cook data class on lines 3-5. The class implements Serializable so it will work with the JVM serialization mechanism. The age property is annotated with @Transient, but more on that later.

The main method creates a tiredBob variable on line 10. On line 14, we open the file bob.ser by passing the name of the file to the FileOutputStream object. The FileOutputStream object is then passed to the constructor of the ObjectOutputStream object. We apply the use() function to make sure the file is closed when finished. The tiredBob object is written to disk by invoking the writeObject() method on the ObjectOutputStream object.

Restoring Bob is done on lines 20-30. Once again, we start by opening the bob.ser file by creating a FileInputStream object and passing the name of the file the constructor. The FileInputStream object is passed to the constructor of the ObjectInputStream object. We chain this operation with the use() function again to ensure the file is closed.

Line 21 restores Bob to memory by calling readObject(). The return type of readObject() is Any, so it’s up to us to down cast the object back into Cook. We do this on lines 23-28. On line 26, we print Bob to the console, but his age is 0. That’s because the age property is transient and therefore excluded from the serialization mechanism.

Transient Properties

Transient properties are defined by adding the @Transient annotation in front of the property (line 5). It’s used because the JVM only serializes primitives and Serializable objects. I once ran into a program where a class was mixed with Swing UI components and the state of the object needed to get sent over network sockets. The UI components were not serializable, so they were marked transient. Otherwise, the JVM would have failed to serialize the object and would have raised an exception.

This was a rather poor design, by the way. The class should have been designed with composition and have the data class separated from the UI components. However, that’s not how this program had been written. Either way, there are certain times where we would want to exclude serializable fields. A password field is a prime example of what should be marked transient.

However, it’s up to the developer to decide what should be included and not included in the serialized object. Kotlin provides the @Transient annotation to accomplish the task. Any object that isn’t @Transient will get included in the serialized object.

Advertisements

Kotlin Object Serialization

Whenever a class implements Serializable, it’s a candidate for object serialization. The serialization mechanism converts an object into bytes and then writes the object to the output stream. We use the class ObjectOutputStream to serialize a file and then ObjectInputStream to restore an object.

import java.io.FileInputStream
import java.io.FileOutputStream
import java.io.ObjectInputStream
import java.io.ObjectOutputStream

fun main(args : Array<String>){
    //Destination File
    val file = "belchers.burgers"

    //A map of family
    val family = mapOf(
            "Bob" to "Father",
            "Linda" to "Mother",
            "Tina" to "Oldest",
            "Gene" to "Middle",
            "Louise" to "Youngest")

    //Write the family map object to a file
    ObjectOutputStream(FileOutputStream(file)).use{ it -> it.writeObject(family)}

    println("Wrote $file")
    println()
    println("Time to read $file back")

    //Now time to read the family back into memory
    ObjectInputStream(FileInputStream(file)).use { it ->
        //Read the family back from the file
        val restedFamily = it.readObject()

        //Cast it back into a Map
        when (restedFamily) {
            //We can't use <String, String> because of type erasure
            is Map<*, *> -> println(restedFamily)
            else -> println("Deserialization failed")
        }
    }
}

The example program writes a map of strings to a file using object serialization. It begins by creating a map of test data on lines 11-16. Line 19 opens the file by creating a FileOutputStream object and passing in the file name to the constructor. The FileOutputStream object gets passed to the newly created ObjectOutputStream. We apply the use() function to make sure all resources are closed when finished.

Writing the map to the file is painless. All we need to do is use the writeObject() method found on ObjectOutputStream, shown on line 19. The class does all of the work of flattening the family Map object into bytes and writing the bytes to the file. The use() function closes the file and the serialization process is complete.

Reading the object back into memory is almost as simple. We open the file by creating a new FileInputStream object and supplying the constructor with the file name. The FileInputStream object is supplied to the constructor of the ObjectInputStream and we chain it to the use() function to make sure the file gets closed when finished.

The object is restored with the readObject() method, but there is a catch. The readObject() method returns Any. It’s our job to downcast to the proper type. On line 31, we use the when() function and on line 33, we check that it is a Map. Since map is a generic interface and serialization doesn’t save type, we use *, * for the type arguments. At this point, we can work on the restedFamily object normally.

Kotlin Data Streams

Data streams are used to write binary data. The DataOutputStream writes binary data of primitive types, while DataInputStream reads data back from the binary stream and converts it to primitive types. Here is an example program that writes data to a file and then reads it back into memory.

import java.io.DataInputStream
import java.io.DataOutputStream
import java.io.FileInputStream
import java.io.FileOutputStream

fun main(args : Array<String>){
    val burgers = "data.burgers"

    //Open the file in binary mode
    DataOutputStream(FileOutputStream(burgers)).use { dos ->
        with(dos){
            //Notice we have to write our data types
            writeInt("Bob is Great\n".length) //Record length of the array
            writeChars("Bob is Great\n") //Write the array
            writeBoolean(true) //Write a boolean

            writeInt("How many burgers can Bob cook?\n".length) //Record length of array
            writeBytes("How many burgers can Bob cook?\n") //Write the array
            writeInt(Int.MAX_VALUE) //Write an int

            for (i in 0..5){
                writeByte(i) //Write a byte
                writeDouble(i.toDouble()) //Write a double
                writeFloat(i.toFloat()) //Write a float
                writeInt(i) //Write an int
                writeLong(i.toLong()) //Write a long
            }
        }
    }

    //Open a binary file in read mode. It has to be read in the same order
    //in which it was written
    DataInputStream(FileInputStream(burgers)).use {dis ->
        with (dis){
            val bobSize = readInt() //Read back the size of the array
            for (i in 0 until bobSize){
                print(readChar()) //Print the array one character at a time
            }
            println(readBoolean()) //Read a boolean

            val burgerSize = readInt() //Length of the next array
            for (i in 0 until burgerSize){
                print(readByte().toChar()) //Print array one character at a time
            }
            println(readInt()) //Read an int

            for (i in 0..5){
                println(readByte()) //Read a byte
                println(readDouble()) //Read a double
                println(readFloat()) //Read a float
                println(readInt()) //Read an int
                println(readLong()) //Read a long
            }
        }

    }
}

The program creates a FileOutputStream object and passes the name of the file to its constructor. The FileOutputStream object is then passed to the constructor of DataOutputStream. We apply the use() function to ensure all resources are freed properly when we have finished. The file is now open for writing in binary mode.

When we wish to use the same object repeatedly, we can pass it to the with() function. In our case, we intend to keep using our DataOutputStream object, so on line 11, we pass it to the with() function. Inside of the with() function, all method calls will target the dos object because it was supplied to with().

Since we intend to write a string to the file, we need to record the length of the string. We do this using the writeInt function and passing the length of our string to it. Then we can use writeChars() to write a character array to the file. The String argument is converted to a character array and written to the file. Finally, we call writeBoolean to write true/false values to the file.

The next section is a repeat of the first. We intend to write another string to the file, but do so, we need to record the length of the file. Once again, we turn to writeInt() to record an int value. The next line, we use writeBytes() rather than writeChars() to demonstrate how we can write a byte array rather than a String. The DataOutputStream class sees to the details of turning a String into a byte array. Finally, we write another int value to the stream.

Next, we enter a for loop on line 21. Inside of the for loop, we demonstrate writing different primitive types to the file. We can use writeByte() for a byte, writeDouble() for a double, and so on for each primitive type. The DataOutputStream class knows the size of each primitive type and writes the correct number of bytes for each primitive.

When we are done writing the object, we open it again to read it. Line 33 creates a FileInputStream object that accepts the path to the file in its constructor. The FileInputStream object is chained to DataInputStream by passing it to the constructor of DataInputStream. We apply the use() function to ensure all resources are properly closed.

Reading the file requires the file to be read in the same order in which it is written. Our first order of business is to grab the size of the character array we wrote to the file earlier. We use readInt() on line 35 followed by a for loop that terminates at the size of the array on line 36. Each iteration of the for loop calls readChar() and the String is printed to the console. When we are finished, we read a boolean on line 39.

Our next array was a byte array. Once again, we need it’s final size so we call readInt() on line 41. Lines 42-44 run through the array and call readByte() until the loop terminates. Each byte is converted to a character object using toChar(). On line 45, we read an int using readInt().

The final portion of the program repeats the for loop found earlier. In this case, we enter a for loop that terminates after five iterations (line 47). Inside of the for loop, we call readByte(), readDouble(), readFloat(), and so on. Each call prints the restored variable to the console.

Kotlin Scanner

The Scanner class is a powerful class that looks for tokens in an input stream and returns each match. The class is often used on files, but it can work with other strings, network sockets, or just about any other character input stream object. The following program demonstrates using a Scanner object to search for words without punctuation. It reads a file and then outputs the most frequently used words to the least frequently used words.

import java.io.FileReader
import java.util.*

fun main(args : Array<String>){
    //Check if they supplied a file
    if(args.isEmpty()){
        println("Please provide a file")
        System.exit(-1)
    }

    //Create an empty map
    val wordMap = mutableMapOf<String, Int>()

    //Open the file and pass it to a Scanner object.
    Scanner(FileReader(args[0])).use { sc ->

        //Tell the scanner to only match entire words
        sc.useDelimiter("""\W""".toPattern())

        //Loop until we get to the end of the file
        while(sc.hasNext()){

            //Grab the next word
            val word = sc.next()

            //Test that it's not a blank string
            if(word.isNotBlank()){
                //Add it to the word map
                wordMap[word] = wordMap.getOrDefault(word, 0) + 1
            }
        }
    }

    //This prints the entries by most used words to least used words
    wordMap.entries.sortedByDescending { it.value }.forEach({it -> println(it)})
}

The program starts by checking if the user provided command line arguments. If the args array is empty, the program exits after printing an error message. Line 12 creates an empty mutable map so that we can add words and counts to it. Individual words are used as the key while the Int is used for values to represent the number of times the word is found.

The file is opened on line 15. We create a FileReader object and pass the path of the file to its constructor. The file path is found at the first element in the arguments array and was supplied by the user. The FileReader object is passed to the constructor of the Scanner. We apply the use() function to ensure the Scanner and the underlying file is closed when have finished.

Line 18 tells the Scanner to match whole words by passing in a regex string and converting it to a Pattern object. The regex “\W” matches whole words. Kotlin allows use to use raw strings inside of triple quotes “”” so that we do not need to worry about escaping any characters.

Line 21 enters a while loop that terminates when Scanner.hasNext() is false. That means we loop until there are no more matches in the input stream. Line 27 tests if the word is a blank string and if it isn’t a blank string, we update the word count on line 29.

Line 35 prints each word from the most used to the least used. It’s accomplished by getting the entries list and then sorting it in descending order. The sortedByDescending takes a comparator object which is created by the lambda expression it.value. In this case, it.value represents the number of times a word was found. The final forEach() operation iterates through the sorted list of entries and prints them individually to the console.

Here was my output when I used this program with a brief excerpt from Green Eggs and Ham.

I=37
not=29
like=28
them=28
do=20
a=19
am=10
Sam=10
you=8
would=7
eggs=6
and=6
ham=6
Would=6
In=6
in=6
or=5
there=5
eat=5
Not=5
Here=4
house=4
mouse=4
with=4
You=4
That=3
Green=3
With=3
green=3
box=3
fox=3
car=3
Anywhere=2
here=2
anywhere=2
Eat=2
could=2
may=2
tree=2
Do=1
Could=1
they=1
are=1
will=1
see=1
let=1
me=1
be=1

Kotlin Reader Example

The java.io.Reader class provides a low-level API for reading character streams.

import java.io.FileReader

fun main(args : Array<String>){
    if (args.isEmpty()){
        println("Please provide a list of files separated by spaces")
        System.exit(-1)
    }

    //Read each supplied file
    args.forEach { fileName ->

        //Open the file. The use() extension function sees to the details
        //of closing the file when finished
        FileReader(fileName).use {

            //Read a single character
            var character = it.read()

            //read() returns -1 at End of File
            while (character != -1){

                //Print the character (make sure to convert it to a Character)
                print(character.toChar())

                //Read the next character
                character = it.read()
            }
        }
    }
}

The example program requires names of text files passed in as command line arguments so our first task is to check if we have any command line arguments. On line 4, we use the isEmpty() function on the args array object to check for an empty array. If true, we print a message to the user (line 5) and then exit the program (line 6).

Provided the program is still running, we begin by printing the contents of each file to the console. On line 10, we enter into a forEach statement to process each of the file supplied at the command line. Rather than using the standard it varaible name, we use fileName to help make the code more clear.

Line 14 performs the operation of actually opening the file. We do this by creating a new FileReader object and pass the name of the file into its constructor. Then we chain the object to the use() extension function. The use() function sees to the details of actually closing the file when we are finished with it, even in the case of an exception.

The file reader object is now referred to by the variable it. On line 17, we call it.read() to read a single character from the file and store it into the character variable. We then enter into a while loop that terminates when character is -1. The -1 value indicates we have reached the end of the file. Inside of the while loop, we print the character (line 23). Sicne read() returns an int, we have to call toChar() to print the actual character. Then on line 26, we update character to the next character in the stream.

Here is how I ran the program for those readers who wish to try it out.

kotlinc ReaderExample.kt -include-runtime -d readerExample.jar
java -jar readerExample.jar ReaderExample.kt

This invocation printed the example program to my console, but it works with any text file.

Python Line Scanner

This post borrows from a code example found in Programming Python: Powerful Object-Oriented Programming that demonstrates collecting command line arguments, opening a file, reading the file, and passing a function as a callback to another function.

Code

Here is the entire script that accepts a file as a command line argument and prints the contents of the file to the console.


def scanner(name, func):

    # Open the file (with statement ensures closure even if there is an exception)
    with open(name, 'r') as f:
        # Iterate through the file
        for line in f:
            # Call our callback function
            func(line)

if __name__ == '__main__':
    import sys
    name = sys.argv[1]

    # This is a function we are passing to scanner
    # Python has first class functions which can be
    # get passed as arguments to other functions
    def print_line(str):
        print(str, end='')

    # Call the scanner function, which in turn
    # calls the print_line function for each line
    # in the file
    scanner(name, print_line)

Command Line Arguments

The first concept covered in this script is processing command line arguments. Python requires us to import the sys module (line 12) which maintains an argv property. The argv property is a list-like object that contains all of the command line arguments used to hold all of the command line parameters. The first index [0] is the name of the script, followed by all of the other arguments supplied to the program.

On line 13, we grab the target file (stored in argv[1]) and keep it in a name variable. At this point, our program knows which file to the open later on when we use the scanner function.

First Class Functions

Python treats functions as objects. As such, we can define any function in a Python program and store it in a variable just like anything else. Lines 18-19 define a print_line function that accepts a String parameter. On line 24, print_line is the second argument to the scanner function.

Once inside of the scanner function, the print_line function is referenced by the variable func. On line 9, we call print_line with the func(line) rather than print_line(line). This works because func and print_line both refer to the same function object in memory. Passing functions in this fashion is incredibly powerful because it allows the scanner function to accept different behaviors for each line it processes.

For example, we could define a function the writes each line processed by scanner to a file rather than printing it to the console. Later on, we may choose to write another function that sends each line over the network via network sockets. The beauty of the scanner function as defined is that it works the same regardless of the callback function passed to the func argument. This programming technique is sometimes known as programming to a behavior.

Opening and Reading Files

The final topic covered is opening and reading a file. Line 5 in the script uses the with statement combined with the open function to actually open the file in read mode. The as f assigns the result of the open function to the variable f. The f variable holds a Python file object.

Since Python file objects support the iterator protocol, they can be used in for loops. On line 7, we read through each line in the file with the statement for line in f:. On each execution of the loop, the line variable is updated with the next line in the file.

When the loop is complete, the with statement calls the file’s close() method automatically, even if there is an exception. Of course, Python’s garabage collection will also ensure a file is closed, but this pattern provides an extra level of safety, especially since there are a variety of Python interpretors that may act differently than the CPython.

Conclusion

The most powerful take away from this example if the first class functions. Python treats functions like any other data type. This allows functions to be stored as passed around the program as required. Using first class functions keeps code loosely coupled and highly maintanable!

Sources

Lutz, Mark. Programming Python. Beijing, OReilly, 2013.