Spring Boot Caching with Kotlin

It’s fairly common for applications to continually ask a datastore for the same information repeatedly. Requests to datastores consume application resources and thus have a performance cost even when the requested data is small. The Spring Platform provides a solution allows applications to store information in an in memory caching system that allows applications to check the cache for the required data prior to making a call to the database. This example shows how to use Spring Boot and Kotlin to cache files that we are storing in the database.

Database Entity

We are going to define a database entity that stores files in a database. Since retrieving such data can be an expesive call to the database, we are going to cache this entity.

@Entity
data class PersistedFile(
        @field: Id @field: GeneratedValue var id : Long = 0,
        var fileName : String = "",
        var mime : String = "",
        @field : Lob var bytes : ByteArray? = null)

You will notice that this class has a ByteArray field that is stored as a LOB in the database. In theory, this could be as many bytes as the system allows so ideally we would store this in cache. Other good candidates are entity classes that have complex object graphs and may result in the ORM generated complex SQL to retreive the managed object.

Enable Caching

Spring Boot defines a CachingManager internally for the application. You are free to use your own, but you need to configure your Spring Boot environment first.

Dependencies

You need to have spring-boot-starter-cache in your pom.xml or other dependency manager.


    org.springframework.boot
    spring-boot-starter-web

Annotation

You also need to tell the environment to turn on caching by using the @EnableCaching

@SpringBootApplication
@EnableJpaRepositories
@EnableCaching  //Spring Boot provides a CacheManager our of the box
                //but it only turns on when this annotation is present
class CachingTutorialApplication

Decorate the Caching Methods

At this point, we only need to decorate the methods we want the environment cache. This is done by decorating our methods with the @Cacheable annotation and then providing the annotation with the name of a cache. We can also optionally tell the cache manager what to use for the key. Here is the code for our service class followed by an explanation.

//We are going to use this class to handle caching of our PersistedFile object
//Normally, we would encapsulate our repository, but we are leaving it public to keep the code down
@Service
class PersistedFileService(@Autowired val persistedFileRepository: PersistedFileRepository){

    //This annotation will cause the cache to store a persistedFile in memory
    //so that the program doesn't have to hit the DB each time for the file.
    //This will result in faster page load times. Since we know that managed objects
    //have unique primary keys, we can just use the primary key for the cache key
    @Cacheable(cacheNames = arrayOf("persistedFile"), key="#id")
    fun findOne(id : Long) : PersistedFile = persistedFileRepository.findOne(id)

    //This annotation will cause the cache to store persistedFile ids
    //By storing the ids, we don't need to hit the DB to know if a file exists first
    @Cacheable(cacheNames = arrayOf("persistedIds"))
    fun exists(id: Long?): Boolean = persistedFileRepository.exists(id)
}

The first method, findOne, is used to look up a persistedFile object from the database. You will notice that we pass persistedFile as an argument to cacheNames and then use the primary key as the key for this item’s cache. We can use the PK because we know it’s a unique value so we can help make the cache more performant. However, keep in mind that the key is optional.

We can also avoid another call to the database by storing if items exist in the database in the cache. The first time exists() is called, the application will fire a count sql statement to the database. On subsequent calls, the cache will simply return true or false depending on what is stored in the cache.

Putting it all together

I put together a small web application that demonstates the caching working together. I turned on the show sql property in the applications.properties file so that viewers can see when the application is making calls to the database. You will notice that the first time I retreive the persisted file, there is sql generated. However, on the second call to the same object, no sql is generated because the application isn’t making a call to the database.

You can get the complete code from my GitHub page at this link.

Here are some links to posts that are related to concepts used in Spring Boot that we used today.

Advertisements

Spring Boot Kotlin & MongoDB

MongoDB is a NoSQL database that works really well with Kotlin and Spring Boot. MongoDB is incredibly useful in situations where the structure of data isn’t known prior to writing the application. For example, picture a blogging website where users can enter any number of comments or response. Modeling such a data structure would be difficult in a relational database, but it’s much easier with Mongo.

In this example application, we are going to use MongoDB to document Restaurants with any number of employees (of course, a simple example such as this can be done in a relational database, but let’s go with this for simplicity sake). The cool part using Mongo with Spring Boot is that there is zero configuration providing you are using default settings. This let’s us jump right into our code.

Let’s begin by creating a couple of data classes to store in our database.

//Create a document class
//that persists to the DB
@Document
data class Restaurant(
        //Mark this field as the document id
        @field: Id var name : String = "",
        //Unstructured Data Here
        var employees : List = mutableListOf())

//This class embeds directly into Restaurant
//without any annotations
data class Employee(var name : String = "",
                    var position : String = "")

Our Restaurant class is annotated with @Document to mark it as a persistable class. We also annotate the name field with the Id annotation to mark it as the document id. This value has to be unique in the database. The other class is Employee which does not have any annotations at all. It’s used as a property in the Employees database and the persistence provide is able store all of employee objects embedded in Restaurant.

Our next class is a repository class which Spring will generate the implementation for us. Before this can happen, we have to enable mongo repositories. All we need to do is annotate a configuration class to make this happen.

@Configuration
@EnableMongoRepositories //Allow Spring to Generate Mongo Repositories
class Config

Once we have enabled the mongo repositories, we just need to define an interface that extends MongoRespository.

//Spring will implement our interface for us!
interface RestaurantRepository : MongoRepository

Now let’s make a controller class to test our application. See this post for an explanation of Spring MVC.

//Example Controller class for demonstration purposes
@Controller
@RequestMapping("/")
class IndexController(
        //We can inject our RestaurantRepository class, Spring will
        //provide an implementation
        @Autowired private val restaurantRepository: RestaurantRepository){

    @RequestMapping(method = arrayOf(RequestMethod.GET))
    fun doGet(model : Model) : String {
        model.apply {
            addAttribute("restaurant", Restaurant())
            //Query all Restaurants
            addAttribute("allRestaurants", restaurantRepository.findAll())
        }
        return "index"
    }

    @RequestMapping(method = arrayOf(RequestMethod.POST))
    fun doPost(@RequestParam("name") name : String,
               @RequestParam("employees") employees : String,
               model : Model) : String {
        val restaurant = Restaurant(name = name,
                                    employees = parseEmployees(employees))
        //Save the new restaurant
        restaurantRepository.save(restaurant)
        model.apply {
            addAttribute("restaurant", Restaurant())
            //Query all Restaurants
            addAttribute("allRestaurants", restaurantRepository.findAll())
        }
        return "index"
    }

    fun parseEmployees(employees : String) : List {
        val employeeList = mutableListOf()
        val parts = employees.split('\n')

        parts.forEach {
            val subParts = it.split(",")
            employeeList.add(
                    Employee(name = subParts[0],
                            position = subParts[1]))
        }
        return employeeList.toList()
    }
}

Notice that we can directly inject RestaurantRepository into our controller. Spring does the work of providing an implementation for our controller class. In our doPost() method, we call restaurantRepository.save() to save our new document. In both doGet() and doPost(), we call restaurantRepository.findAll() to pull back all of our restaurants stored in the database.

Now we just need an HTML template to provide us with front end code.
indexcode

Conclusion

Here is an example of the application when run.


As you can see, Spring Boot combined with Kotlin makes it really easy to persist data into MongoDB. We only need to define a few data classes and allow Spring to make our Repository classes for us in order to get started.

You can view the code for this project at my GitHub page at this link.

Kotlin Koans—Part 23

This portion of the Kotlin Koans tutorial appeared to be a review of the concepts I had been working on throughout the collection section. I had to solve three different problems using the collections API. While doing this, I got to revist the Elivis operator (?:), map, maxBy, sumBy, filter, count, and toSet.

Get Customers Who Ordered Product

This problem focused on filtering.

fun Shop.getCustomersWhoOrderedProduct(product: Product): Set {
    // Return the set of customers who ordered the specified product
    return customers.filter { it.orderedProducts.contains(product) }.toSet()
}

The filter method takes a predicate that returns true or false. In this case, I just used the contains method on orderedProducts. If the product is found in orderedProducts, we get a true, otherwise false. Then there is a toSet() operation to transform the collection to a set.

Get Most Expensive Delived Products

This problem was a little more challenging. I had to go back and review how to use the Elivis operator (TODO: Link).

fun Customer.getMostExpensiveDeliveredProduct(): Product? {
    // Return the most expensive product among all delivered products
    // (use the Order.isDelivered flag)
    return orders.filter { it.isDelivered }.map { it.products.maxBy { it.price } }.maxBy { it?.price ?: 0.0}
}

I started with a filter operation to check if an order was delivered or not since the problem statement required me to find the most expensive delivered product. Then I had to use a map operation which allowed me to traverse all delivered orders. At this point, I could use a maxBy operation and check it.price. This builds up a collection of products that contains the most expensive product on each order.

The next part of the operation is to find the most expensive product of all orders. At this point, I have a collection of products so I just needed another maxBy operation. However it was a little more trickey this time. In this case, there was a possibily that the variable it could be null. It’s nice that Kotlin has compiler checks for this sort of thing because I truthfully didn’t realize that I could be working with null objects here. Thus, I had to use the Elvis operator in this final lambda operation.

Get Number Of Times Product Was Ordered

I had to solve this problem by chaining transformations together again.

fun Shop.getNumberOfTimesProductWasOrdered(product: Product): Int {
    // Return the number of times the given product was ordered.
    // Note: a customer may order the same product for several times.
    return customers.sumBy { it.orders.sumBy { it.products.count { it == product } } }
}

A customer has a one to many relationship with orders, and orders have a one to many relationship with products. I needed two sumBy operations to solve this problem. I began with a sumBy on customers. Inside of the lambda, I did another sumBy operation on orders. Once I was traversing orders, I could do a count operation on products and get a total of how many products matched my predicate.

The it.products.count returns a number that gets fed into it.orders.sumBy. The it.orders.sumBy returns a number that gets fed into customers.sumBy. Once customers.sumBy returns, we have a count of the total number of times the specified product was ordered.

You can click here to see Part 22

Kotlin Spring Security Custom Login

Spring Security provides a custom login page that is functional but not very attractive. In most cases, web developers want a more attractive looking login page. This post demonstrates how to configure Spring Security to use a custom login page. Readers can view this tutorial for a demonstration on how to configure basic Spring Security.

Front End—Write a Custom Page

We are going to start by writing a custom login page. Spring Security is very flexible about the page itself, but there are a few rules that need to be followed.

  • Form requires Username
  • Form requires Password
  • CSRF Token is required

This is a screen shot of the page that will be used for this tutorial.
LoginPage
Followed by the code.
LoginCode
It’s critical to remember to include the CSRF token in the form. Without it, the server will refuse any HTTP POST requests and will respond with code 405. The th:name="${_csrf.parameterName}" th:value="${_csrf.token}" on an input take will do the job of adding the CSRF token.

Backend—Configure Spring

Once the front end code is ready, you need to configure Spring Security to use this page. In order to render the login page, Spring will need some sort of controller class. Developers are free to write their own, but it’s also trivial to make use of the one Spring is happy to provide.

@Configuration
class WebConfig : WebMvcConfigurerAdapter() {
    override fun addViewControllers(registry: ViewControllerRegistry) {
        //This class adds a default controller for the login page.
        //Otherwise you would need to write a custom controller class
        registry.addViewController("/login").setViewName("login")
    }
}

The next job is to configure Spring security.

@Configuration //Make this as a configuration class
@EnableWebSecurity //Turn on Web Security
class SecurityWebInitializer : WebSecurityConfigurerAdapter(){
    override fun configure(http: HttpSecurity) {
        http
                .authorizeRequests()
                    //We need to allow anonymous users to
                    //access the login page (otherwise we get 403)
                    .antMatchers("/login").anonymous()
                    .anyRequest().authenticated()
                .and()
                    //Setup a custom login page
                    .formLogin()
                        .loginPage("/login")
                        .usernameParameter("username")
                        .passwordParameter("password")
                .and()
                    .httpBasic()
    }

    override fun configure(auth: AuthenticationManagerBuilder) {
        //This code sets up a user store in memory. This is
        //useful for debugging and development
        auth
                .inMemoryAuthentication()
                    .withUser("bob")
                    .password("belcher")
                    .roles("USER")
                .and()
                    .withUser("admin")
                    .password("admin")
                    .roles("USER", "ADMIN")
    }
}

It’s important to allow anonymous() access to the login page. Without it, Spring Security will continue to redirect to the login page until the server returns 403 (too many redirects).

Conclusion

Once complete, the site will render a custom login page like what is shown in the video.

You can get the code for the complete project at my GitHub page.

Kotlin Koans—Part 22

More functional programming on the horizon. This portion of Kotlin Koans demonstrated folding. I personally had never tackled a challenge like this so it took me more time to figure it out than the other problems. My job was to go through all customers and the products they ordered and reduce them down to a single set of objects. Here is the Kotlin code.

fun Shop.getSetOfProductsOrderedByEveryCustomer(): Set {
    // Return the set of products ordered by every customer
    return customers.fold(allOrderedProducts, {
        orderedByAll, customer ->
            orderedByAll.intersect(customer.orderedProducts)
    })
}

As usual, I tried to do the same problem in Java for comparison purposes, but I wasn’t able to figure it out! (If you know the solution, please leave it in the comments section!). I’ll have to admit that I am weak in some of the functional programming areas.

You can click here to see Part 21.

Spring Security Form Login with JDBC – Kotlin

Spring Security makes it really simple to authenticate users against a database. This tutorial builds on the previous tutorial of configuring Spring Security to secure web applications.

Database Schema

Spring Security is happy to do all of the work of querying a database and validating user information provided your database conforms to the correct database schema (note, you are free to customize). Here is the sql script that is used to configure an example datasource for this project that is based of the one provided in the Spring documetation.

/* See https://docs.spring.io/spring-security/site/docs/current/reference/html/appendix-schema.html */

DROP TABLE IF EXISTS persistent_logins;
DROP TABLE IF EXISTS group_members;
DROP TABLE IF EXISTS group_authorities;
DROP TABLE IF EXISTS groups;
DROP TABLE IF EXISTS authorities;
DROP TABLE IF EXISTS users;

create table users(
  username varchar_ignorecase(50) not null primary key,
  password varchar_ignorecase(50) not null,
  enabled boolean not null
);

create table authorities (
  username varchar_ignorecase(50) not null,
  authority varchar_ignorecase(50) not null,
  constraint fk_authorities_users foreign key(username) references users(username)
);

create unique index ix_auth_username on authorities (username,authority);

create table groups (
  id bigint generated by default as identity(start with 0) primary key,
  group_name varchar_ignorecase(50) not null
);

create table group_authorities (
  group_id bigint not null,
  authority varchar(50) not null,
  constraint fk_group_authorities_group foreign key(group_id) references groups(id)
);

create table group_members (
  id bigint generated by default as identity(start with 0) primary key,
  username varchar(50) not null,
  group_id bigint not null,
  constraint fk_group_members_group foreign key(group_id) references groups(id)
);

create table persistent_logins (
  username varchar(64) not null,
  series varchar(64) primary key,
  token varchar(64) not null,
  last_used timestamp not null
);

insert into users values('bob_belcher', 'burger_bob', true);
insert into authorities values ('bob_belcher', 'user');

This script drops all tables if they exist and then recreates the database tables. It also populates the database with a user: bob_belcher. Creating and destroying the DB in this fashion is useful for both development purposes and unit testing. Naturally, a production machine would preserve the data each time.

Spring Configuration

Configuring Spring Security to work with our database is a complete breeze at this point. We start by creating two bean definitions for both a data source and a jdbcTemplate.

@Configuration
class DataConfig {

    @Bean(name = arrayOf("dataSource"))
    fun dataSource() : DataSource {
        //This will create a new embedded database and run the schema.sql script
        return EmbeddedDatabaseBuilder()
                .setType(EmbeddedDatabaseType.HSQL)
                .addScript("schema.sql")
                .build()
    }

    @Bean
    fun jdbcTemplate(@Qualifier("dataSource") dataSource: DataSource) : JdbcOperations {
        return JdbcTemplate(dataSource)
    }
}

Since I am using Spring Boot, I did qualify our dataSource bean so that the container knew which bean I wanted to use for our datasource.

Now that we have our data source configured, we just need to tell Spring Security about it. It’s not very difficult.

@Configuration //Make this as a configuration class
@EnableWebSecurity //Turn on Web Security
class SecurityWebInitializer(
        //Inject our datasource into this class for the AuthenticationManagerBuilder
        @Autowired @Qualifier("dataSource") val dataSource: DataSource)
    : WebSecurityConfigurerAdapter(){

    override fun configure(http: HttpSecurity) {
        http
                    .formLogin()
                .and()
                    .logout()
                        .logoutSuccessUrl("/")
                .and()
                    .rememberMe()
                        .tokenRepository(JdbcTokenRepositoryImpl())
                            .tokenValiditySeconds(2419200)
                                .key("BurgerBob")
                .and()
                    .httpBasic()
                .and()
                    .authorizeRequests()
                        .antMatchers("/").authenticated()
                        .anyRequest().permitAll()
    }

    override fun configure(auth: AuthenticationManagerBuilder) {
        //As long as our database schema conforms to the default queries
        //we can use jdbcAuthentication and pass in our data source
        //Spring will do the rest of the work for us
        auth.jdbcAuthentication().dataSource(dataSource)
    }
}

In this case, all that is needed is to call auth.jdbcAuthentication().dataSource and pass in our dataSource object. Spring Security takes it from there.

Conclusion

Here is a video of this in action.

You can grab the entire code from my Github page here.

Kotlin Spring Security Tutorial

Just about anybody can appreciate the value of securing a web application. After all, who would do their online banking on an unsecured website? Of course, it’s not just online banking that requires security. Just about any website that has information that requires protecting needs security.

Spring provides web security modules that help us secure our applications. As with everything in Spring, it’s really easy to use an configure.

Define a Security Class

Spring has us extend the WebSecurityConfigurerAdapter class and annotate it with @Configuration and @EnableWebSecurity. Here is an example Kotlin class that enables our web security and forces all requests to the web application to be authenticated.

@Configuration //Make this as a configuration class
@EnableWebSecurity //Turn on Web Security
class SecurityWebInitializer : WebSecurityConfigurerAdapter(){
    override fun configure(http: HttpSecurity) {
        //This tells Spring Security to authorize all requests
        //We use formLogin and httpBasic
        http
                .authorizeRequests()
                    .anyRequest()
                    .authenticated()
                .and()
                    .formLogin()
                .and()
                    .httpBasic()
    }

    override fun configure(auth: AuthenticationManagerBuilder) {
        //This code sets up a user store in memory. This is
        //useful for debugging and development
        auth
                .inMemoryAuthentication()
                    .withUser("bob")
                    .password("belcher")
                    .roles("USER")
                .and()
                    .withUser("admin")
                    .password("admin")
                    .roles("USER", "ADMIN")
    }
}

The first method, configure(http: HttpSecurity) calls methods on the http object. This class has a chaining interface and by calling the proper methods, we can tailor the security configuration to suit our needs. The methods are plain english, so the code ends up being highly self-documenting.

The other configure method accepts an auth: AuthenticationManagerBuild. The auth object is used to configure a data store for users. For the purposes of this post, we are creating an inMemoryAuthentication. This is useful for development and debugging purposes.

The Controller Class

There isn’t anything special about the controller class. That’s a feature of Spring Security. Security is a cross cutting concern which means that the main application code should not have to concern itself with security. Instead, Spring uses Aspect Orientated programming to secure our application.

Sometimes it’s useful to know what user is logged into this system. There is a an example of how to access this information and pass it back to the view. (Readers who are not familiar with Spring MVC can refer here for an example of Spring MVC).

@Controller
@RequestMapping("/")
class IndexController {

    @RequestMapping(method = arrayOf(RequestMethod.GET))
    fun doGet(model : Model) : String {
        //We can access the current user like this
        val authorization = SecurityContextHolder.getContext().authentication

        //Send the user name back to the view
        model.addAttribute("username", authorization.name)
        return "index"
    }
}

The SecurityContextHolder class provides an access point to the current logged in user. Spring calls it an authentication. The object returns contains information about the user such as the user name.

Conclusion

Here is a video of logging into this site in action.

You can get the code from my github page here.