Spring Security @RolesAllowed JSR250 Kotlin

Although Spring Security provides means to secure the web tier using XML markup, it’s also critically important that developers also secure backend method to ensure that methods. This post demosntrates an application in which a developer forgot to secure a web form but luckily the backend code is secured and provides a safe guard against such an error.

Enabling JSR250

Spring Boot takes a declaritive approaching to enabling method security, but we also need to provide it with an authentication manager.

@Configuration
@EnableJpaRepositories
//The next annotation enabled @RolesAllowed annotation
@EnableGlobalMethodSecurity(jsr250Enabled = true)
//We need to extend GlobalMethodSecurityConfiguration and override the configure method
//This will allow us to secure methods
class MethodSecurityConfig : GlobalMethodSecurityConfiguration(){

    override fun configure(auth: AuthenticationManagerBuilder) {
        //In our case, we are going to use an in memory authentication
        configureAuthentication(auth)
    }
}

fun configureAuthentication(auth: AuthenticationManagerBuilder){
    auth
            .inMemoryAuthentication()
            .withUser("bob").password("bob").roles("ADMIN", "USER")
            .and()
            .withUser("gene").password("gene").roles( "USER")
}

We create a class that extends GlobalMethodSecurityConfiguration. We turn the method security on by annotating this class with @EnableGlobalMethodSecurity. By default, Spring uses it’s own @Secured annotation so if we want to use the JSR standard, we need to pass true to the jsr250Enabled annotation. Then our MethodSecurityConfig class needs to override the configure method and add an authentication scheme.

Readers may be wondering what the difference is between @Secured and @RolesAllowed annotations. There doesn’t seem to be much as both annotations seem to do the same thing. There is the possibility that other software libraries may act on @RolesAllowed and if there is such as concern, then use @Secured.

Securing Methods

Once we have enabled method security, we only need to decorate our specific methods. Here is a service class used in the example application.

@Transactional
//This is our class that we are going to secure
class BurgerService(@Autowired val burgerRepository: BurgerRepository){

    @PostConstruct
    fun init(){
        //Just popuplates the DB for the example application
        val burgers = listOf(
                BurgerOfTheDay(name = "New Bacon-ings"),
                BurgerOfTheDay(name = "Last of the Mo-Jicama Burger"),
                BurgerOfTheDay(name = "Little Swiss Bunshine Burger"),
                BurgerOfTheDay(name = "Itsy Bitsy Teeny Weenie Yellow Polka-Dot Zucchini Burger"))
        burgerRepository.save(burgers)
    }

    @PreDestroy
    fun destory(){
        //Clean up the DB when done
        burgerRepository.deleteAll()
    }

    //Any user can add a new BurgerOfTheDay
    @RolesAllowed(value = *arrayOf("USER", "ADMIN"))
    fun saveBurger(burgerOfTheDay: BurgerOfTheDay) = burgerRepository.save(burgerOfTheDay)

    //But only adminstrators get to delete burgers
    @RolesAllowed(value = "ADMIN")
    fun deleteBurger(id : Long) = burgerRepository.delete(id)

    //Any user gets to see our Burgers
    @RolesAllowed(value = *arrayOf("USER", "ADMIN"))
    fun allBurgers() = burgerRepository.findAll()
}

The @RolesAllows annotation takes an array of allowed roles. In our case, we are letting anyone with the USER role to add burgers, but only ADMIN users are allowed to delete burgers. If a user without the ADMIN role attempts to invoke deleteBurger, an AccessDeniedException is thrown.

Catching Security Violations

Kotlin has no concept of checked exceptions, but Java users should note that Spring’s security exceptions are all RuntimeExceptions. If we want to report a security violation back to the user, we need to catch our security exceptions. Here is an example Controller class that handles security violations.

@Controller
class IndexController(
        @Autowired val logger : Logger,
        @Autowired val burgerService: BurgerService) {

    @GetMapping("/")
    fun doGet(model : Model) : String {
        model.addAttribute("burgers", burgerService.allBurgers().toList())
        return "index"
    }

    @PostMapping("/add")
    fun saveBurger(
            @RequestParam("burgerName") burgerName : String,
            model : Model) : String {
        try {
            burgerService.saveBurger(BurgerOfTheDay(name=burgerName))
            model.addAttribute("burgers", burgerService.allBurgers().toList())
            model.addAttribute("info", "Burger has been added")
        } catch (e : Exception){
            when (e){
                is AccessDeniedException -> {
                    logger.info("Security Exception")
                }
                else -> logger.error(e.toString(), e)
            }
        } finally {
            return "index"
        }
    }

    @PostMapping("/delete")
    fun deleteBurgers(
            @RequestParam("ids") ids : LongArray,
                      model: Model) : String {

        var errorThrown = false

        ids.forEach {
            try {
                burgerService.deleteBurger(it)

                //If the user doesn't have permission to invoke a method,
                //we will get AccessDeniedException which we handle and notify the user of the error
            } catch (e : Exception){
                when (e) {
                    is AccessDeniedException -> {
                        model.addAttribute("error", "Only Bob gets to delete burgers!")
                        logger.info("Security error")
                    }
                    else -> logger.error(e.toString(), e)
                }
                errorThrown = true
            }
        }
        model.addAttribute("burgers", burgerService.allBurgers().toList())
        if(!errorThrown){
            model.addAttribute("info", "Deleted burgers")
        }
        return "index"
    }
}

You’ll ntoice that the deleteBurgers method looks for AccessDeniedException (which is handled by Koltin’s powerful when block). In our case, we report an error that only Bob get’s to delete burgers.

Putting it all together

Here is a video of a sample web application that demonstrates this code in action.

The code for the example application is available at my GitHub page.

You can also learn more about Spring MVC by referring to the following posts.

Spring Boot Caching with Kotlin

It’s fairly common for applications to continually ask a datastore for the same information repeatedly. Requests to datastores consume application resources and thus have a performance cost even when the requested data is small. The Spring Platform provides a solution allows applications to store information in an in memory caching system that allows applications to check the cache for the required data prior to making a call to the database. This example shows how to use Spring Boot and Kotlin to cache files that we are storing in the database.

Database Entity

We are going to define a database entity that stores files in a database. Since retrieving such data can be an expesive call to the database, we are going to cache this entity.

@Entity
data class PersistedFile(
        @field: Id @field: GeneratedValue var id : Long = 0,
        var fileName : String = "",
        var mime : String = "",
        @field : Lob var bytes : ByteArray? = null)

You will notice that this class has a ByteArray field that is stored as a LOB in the database. In theory, this could be as many bytes as the system allows so ideally we would store this in cache. Other good candidates are entity classes that have complex object graphs and may result in the ORM generated complex SQL to retreive the managed object.

Enable Caching

Spring Boot defines a CachingManager internally for the application. You are free to use your own, but you need to configure your Spring Boot environment first.

Dependencies

You need to have spring-boot-starter-cache in your pom.xml or other dependency manager.


    org.springframework.boot
    spring-boot-starter-web

Annotation

You also need to tell the environment to turn on caching by using the @EnableCaching

@SpringBootApplication
@EnableJpaRepositories
@EnableCaching  //Spring Boot provides a CacheManager our of the box
                //but it only turns on when this annotation is present
class CachingTutorialApplication

Decorate the Caching Methods

At this point, we only need to decorate the methods we want the environment cache. This is done by decorating our methods with the @Cacheable annotation and then providing the annotation with the name of a cache. We can also optionally tell the cache manager what to use for the key. Here is the code for our service class followed by an explanation.

//We are going to use this class to handle caching of our PersistedFile object
//Normally, we would encapsulate our repository, but we are leaving it public to keep the code down
@Service
class PersistedFileService(@Autowired val persistedFileRepository: PersistedFileRepository){

    //This annotation will cause the cache to store a persistedFile in memory
    //so that the program doesn't have to hit the DB each time for the file.
    //This will result in faster page load times. Since we know that managed objects
    //have unique primary keys, we can just use the primary key for the cache key
    @Cacheable(cacheNames = arrayOf("persistedFile"), key="#id")
    fun findOne(id : Long) : PersistedFile = persistedFileRepository.findOne(id)

    //This annotation will cause the cache to store persistedFile ids
    //By storing the ids, we don't need to hit the DB to know if a file exists first
    @Cacheable(cacheNames = arrayOf("persistedIds"))
    fun exists(id: Long?): Boolean = persistedFileRepository.exists(id)
}

The first method, findOne, is used to look up a persistedFile object from the database. You will notice that we pass persistedFile as an argument to cacheNames and then use the primary key as the key for this item’s cache. We can use the PK because we know it’s a unique value so we can help make the cache more performant. However, keep in mind that the key is optional.

We can also avoid another call to the database by storing if items exist in the database in the cache. The first time exists() is called, the application will fire a count sql statement to the database. On subsequent calls, the cache will simply return true or false depending on what is stored in the cache.

Putting it all together

I put together a small web application that demonstates the caching working together. I turned on the show sql property in the applications.properties file so that viewers can see when the application is making calls to the database. You will notice that the first time I retreive the persisted file, there is sql generated. However, on the second call to the same object, no sql is generated because the application isn’t making a call to the database.

You can get the complete code from my GitHub page at this link.

Here are some links to posts that are related to concepts used in Spring Boot that we used today.

Spring Boot Kotlin & MongoDB

MongoDB is a NoSQL database that works really well with Kotlin and Spring Boot. MongoDB is incredibly useful in situations where the structure of data isn’t known prior to writing the application. For example, picture a blogging website where users can enter any number of comments or response. Modeling such a data structure would be difficult in a relational database, but it’s much easier with Mongo.

In this example application, we are going to use MongoDB to document Restaurants with any number of employees (of course, a simple example such as this can be done in a relational database, but let’s go with this for simplicity sake). The cool part using Mongo with Spring Boot is that there is zero configuration providing you are using default settings. This let’s us jump right into our code.

Let’s begin by creating a couple of data classes to store in our database.

//Create a document class
//that persists to the DB
@Document
data class Restaurant(
        //Mark this field as the document id
        @field: Id var name : String = "",
        //Unstructured Data Here
        var employees : List = mutableListOf())

//This class embeds directly into Restaurant
//without any annotations
data class Employee(var name : String = "",
                    var position : String = "")

Our Restaurant class is annotated with @Document to mark it as a persistable class. We also annotate the name field with the Id annotation to mark it as the document id. This value has to be unique in the database. The other class is Employee which does not have any annotations at all. It’s used as a property in the Employees database and the persistence provide is able store all of employee objects embedded in Restaurant.

Our next class is a repository class which Spring will generate the implementation for us. Before this can happen, we have to enable mongo repositories. All we need to do is annotate a configuration class to make this happen.

@Configuration
@EnableMongoRepositories //Allow Spring to Generate Mongo Repositories
class Config

Once we have enabled the mongo repositories, we just need to define an interface that extends MongoRespository.

//Spring will implement our interface for us!
interface RestaurantRepository : MongoRepository

Now let’s make a controller class to test our application. See this post for an explanation of Spring MVC.

//Example Controller class for demonstration purposes
@Controller
@RequestMapping("/")
class IndexController(
        //We can inject our RestaurantRepository class, Spring will
        //provide an implementation
        @Autowired private val restaurantRepository: RestaurantRepository){

    @RequestMapping(method = arrayOf(RequestMethod.GET))
    fun doGet(model : Model) : String {
        model.apply {
            addAttribute("restaurant", Restaurant())
            //Query all Restaurants
            addAttribute("allRestaurants", restaurantRepository.findAll())
        }
        return "index"
    }

    @RequestMapping(method = arrayOf(RequestMethod.POST))
    fun doPost(@RequestParam("name") name : String,
               @RequestParam("employees") employees : String,
               model : Model) : String {
        val restaurant = Restaurant(name = name,
                                    employees = parseEmployees(employees))
        //Save the new restaurant
        restaurantRepository.save(restaurant)
        model.apply {
            addAttribute("restaurant", Restaurant())
            //Query all Restaurants
            addAttribute("allRestaurants", restaurantRepository.findAll())
        }
        return "index"
    }

    fun parseEmployees(employees : String) : List {
        val employeeList = mutableListOf()
        val parts = employees.split('\n')

        parts.forEach {
            val subParts = it.split(",")
            employeeList.add(
                    Employee(name = subParts[0],
                            position = subParts[1]))
        }
        return employeeList.toList()
    }
}

Notice that we can directly inject RestaurantRepository into our controller. Spring does the work of providing an implementation for our controller class. In our doPost() method, we call restaurantRepository.save() to save our new document. In both doGet() and doPost(), we call restaurantRepository.findAll() to pull back all of our restaurants stored in the database.

Now we just need an HTML template to provide us with front end code.
indexcode

Conclusion

Here is an example of the application when run.


As you can see, Spring Boot combined with Kotlin makes it really easy to persist data into MongoDB. We only need to define a few data classes and allow Spring to make our Repository classes for us in order to get started.

You can view the code for this project at my GitHub page at this link.

Kotlin Spring Security Hibernate Login

In a previous post, I showed how we can use Spring Security with JDBC to store user creditionals in a database. This approach works fine in small projects but I find it to be limiting in larger applications. Many larger applications prefer to use some sort of Object Relational Mapping (ORM) library to handle storing mapped objects to a database. This post shows how to configure Spring Security to use Hibernate to look up saved users in a database.

applications.properties

Spring Boot uses an application.properties file to configure the application. By default, Spring Boot provides an embedded datastore for the application. We only need minor configuration to make it work with Hibernate.

spring.jpa.hibernate.ddl-auto=create-drop
spring.jpa.properties.hibernate.current_session_context_class=org.springframework.orm.hibernate5.SpringSessionContext
spring.datasource.driver-class-name=org.hsqldb.jdbcDriver

The first line tells the application to scan any classes marked with the @Entity annotation and create database tables for these objects. The next line configures tells it that we wish to use Hibernate. The final line tells the application which JDBC driver to use to interact with the database.

Mapped Objects

Hibernate (and other ORMS) use decorated objects to map to the database. In many cases, these are simply objects that have a list of fields and getters and setters (POJOs) and overide equals() and hashcode(). In most cases, this ends up causing a lot of boiler plate code. Kotlin provides us with data classes that cut down on the noise.

Roles

Spring Security tracks user roles throughout the application, so we need a class to represent user roles.

@Entity
data class Roles(@field: Id @field: GeneratedValue var id : Int = 0,
                 @field: ManyToOne(targetEntity = SiteUser::class) var user : SiteUser,
                 var role: String = "")

This class defines a POKO (Plain Old Kotlin Object) that represents Roles. It’s very boring, but readers will notice how to annotate fields in Kotlin [@field: [Java Annotation]]. So in the case of @Id, we just use @field: Id. The same holds true for @ManyToOne and other JPA annotations.

SiteUser

Since Spring Security has a User class, I find it to be more readable to name our persistent user as SiteUser.

@FetchProfiles(
        FetchProfile(name = "default",
                fetchOverrides = arrayOf(
                        FetchProfile.FetchOverride(entity = SiteUser::class, association = "roles", mode = FetchMode.JOIN)))
)
@Entity
data class SiteUser (@field: Id @field: GeneratedValue var id : Int = 0,
                     var userName: String = "",
                     var password: String = "",
                     var enabled : Boolean = true,
                     var accountNonExpired: Boolean = true,
                     var credentialsNonExpired: Boolean = true,
                     var accountNonLocked : Boolean = true,
                     @field: OneToMany(targetEntity = Roles::class) var roles: MutableSet = mutableSetOf()){

    //Convert this class to Spring Security's User object
    fun toUser() : User {
        val authorities = mutableSetOf()
        roles.forEach { authorities.add(SimpleGrantedAuthority(it.role)) }
        return User(userName, password,enabled, accountNonExpired, credentialsNonExpired, accountNonLocked,authorities);
    }
}

This is another data class with JPA mappings. The SiteUser class have a one to many relationship to Roles (in other words, one user can have multiple roles). Hibernate lazily loads collections by default, so unless we explicilty tell it to eager fetch our user roles, we will get a proxy error later on the in application.

There are several ways to work around this issue. We can use HQL (Hibernate Query Language) to eagerly load Roles. Another solution it to pass FetchType.Eager argument to the OneToMany annotation. A final approach is the one seen here and that’s to use Fetch Profiles to instruct Hibernate what to load. One advantage of FetchProfiles is that a class can have multiple fetch profiles, so using FetchProfiles is a highly flexible solution.

The other thing to note about this class is how to hooks into Spring Security. This class has a toUser() method which is a utility method that converts our SiteUser object into a Spring Security User object. If you look closely, the fields on our SiteUser class are the exact same fields as the User class. This makes it really easy to convert a SiteUser to a User.

Since we configured the application.properties to generate our database DDL (spring.jpa.hibernate.ddl-auto=create-drop), Hibernate will see to the details of scanning our Roles and User class and generated the necessary database tables for us. There is no further work for us to do at this point regarding the data store.

Data Configuration

Our next job is to provide Spring Security with a path to look up Users from the database.

DataConfig

Spring Security needs a path to the database in order to look up User objects. That means we are going to need Repository and Service classes in the application, but those classes depend on a SessionFactory object from Hibernate.

@Configuration
class DataConfig {
    @Bean
    fun sessionFactory(@Autowired entityManagerFactory: EntityManagerFactory) :
            SessionFactory = entityManagerFactory.unwrap(SessionFactory::class.java)
}

We really just need a bean definition for a SessionFactory. Spring Boot is configured to use JPA (Java Persistence Api), which is the ORM standard that Hibernate and other ORM libraries implement. There two main advantages of using the standard JPA rather than vendor API.

  1. Other developers are likely to know the standard API over vendor specific APIs
  2. You can swap ORM libraries when sticking to the standard

In reality, I have never been on a project that switched ORM libraries and there are times when an ORM library offers features that aren’t offered in a standard. Since we know that we are going to use Hibernate, we can just unwrap the SessionFactory object from the injected entityManagerFactory and just return the SessionFactory. At this point, we can inject SessionFactory into our classes and use Hiberante API directly.

UserRepository

UserRepository works directly with the database.

@Repository
//Inject SessionFactory into this class
class UserRepository(@Autowired private val sessionFactory: SessionFactory){

    //Used to save new users into the datastore
    fun saveOrUpdate(user: SiteUser){
        sessionFactory.currentSession.saveOrUpdate(user)
    }

    //Query the database by user name and return a SiteUser that matches
    //the user name
    fun loadByUsername(userName: String) : SiteUser =
            sessionFactory.currentSession.createCriteria(SiteUser::class.java, "su")
                    .add(Restrictions.eq("su.userName", userName)).uniqueResult() as SiteUser

    //Return all Site Users from the database
    fun allUsers(profile : String = "default") : List {
        val session = sessionFactory.currentSession
        session.enableFetchProfile(profile)
        return session.createCriteria(SiteUser::class.java).list() as List
    }
}

You will notice that we inject SessionFactory into this class. Spring Security needs to query the database by the UserName, so loadByUsername uses Hibernate’s Criteria API to create a query that searches for users that match the user name. The other two methods in this class are not related to Spring Security but are used by the application. The saveOrUpdate() method is used to persist a new user into the databse. The allUsers() method returns all users stored in the database.

UserService

The UserService class provides the glue between Spring Security and the Database.

@Transactional //Have Spring Manage Database Transactions
@Service //Mark this class as a Service layer class
class UserService(@Autowired private val userRepository: UserRepository) //Inject UserRepository into this class
    : UserDetailsService { //To work with Spring Security, it needs to implement UserDetailsService

    //Load a user by user name and call our toUser() method on SiteUser
    override fun loadUserByUsername(userName: String): UserDetails  = userRepository.loadByUsername(userName).toUser()

    //Saves a new user into the datastore
    fun saveOrUpdate(user : SiteUser){
        //Encrypt their password first
        user.password = BCryptPasswordEncoder().encode(user.password)

        //Then save the user
        userRepository.saveOrUpdate(user)
    }

    //Return all users
    fun allUsers() = userRepository.allUsers()
}

Spring provides container managed transactions when a class is marked @Transactional. The important part of this class is that it implements UserDetailsService, which allows this class to get passed to Spring Security when we configure our authentication (next section). The loadByUsername method comes from the UserDetailsService interface. It returns a User object, which means we need to call our toUser() method that we defined on SiteUser() to convert SiteUser() to User().

The other method of interest is the saveOrUpdate() method. You will notice that we encrypt our User’s password prior to saving the object to the database. This is a critical step because without it, anyone could peek into our database and get our users password. We also need to encrypt the passwords because we configure our authentication to decrypt passwords later on.

Configuring Spring Security

Now that we have a path that allows the application to access and retreive users from the databse, we are ready to configure Spring Security.

SecurityConfig

The SecurityConfig class does the work of configuring our Spring Security in this application.

@Configuration
class SecurityConfig(@Autowired private val userService : UserService) : //Inject UserService
        WebSecurityConfigurerAdapter() { //Extend WebSecurityConfigureAdaptor

    //Override this method to configure Authentication
    override fun configure(auth: AuthenticationManagerBuilder) {
        auth
                .userDetailsService(userService) //We pass our userService to userDetailsService
                .passwordEncoder(BCryptPasswordEncoder()) //Pass our Encryption Scheme also
    }

    override fun configure(http: HttpSecurity) {
        http.
                formLogin()
                    .and()
                .httpBasic()
                    .and()
                .authorizeRequests()
                    .antMatchers("/display").authenticated()
                    .anyRequest().permitAll()

    }
}

The configure(AuthenticationManagerBuilder) is our method of interest. The auth object has a userDetailsService method that accepts any class that implements UserDetailsService. Since our UserService class implements this interface, it can be used as a value for the userDetailsService method. At that point, our Security is linked to our database. The other method is passwordEncoder that takes an instance of ByCryptPasswordEncorder(), the same class used UserService to encrypt our passwords. Now the AuthenticationManagerBuilder can speak to our database and decode our passwords.

Controller Class

At this point, our application is configured to work Spring Security and a database. Our next two classes setup Spring MVC so that we have a working example.

RegisterController

RegisterController is used to add users to the application.

@Controller
@RequestMapping("/register")
class RegisterController(@Autowired private val userService: UserService){

    @RequestMapping(method = arrayOf(RequestMethod.GET))
    fun doGet(model: Model) : String{
        model.addAttribute("user", SiteUser())
        return "register"
    }

    @RequestMapping(method = arrayOf(RequestMethod.POST))
    fun doPost(siteUser: SiteUser) : String{
        userService.saveOrUpdate(siteUser)
        return "redirect:/display"
    }
}

UserDisplay

UserDisplay controls the display page and shows all users in our database.

@Controller
@RequestMapping("/display")
class UserDisplay(@Autowired private val userService: UserService){

    @RequestMapping(method = arrayOf(RequestMethod.GET))
    fun doGet(model: Model) : String{
        model.addAttribute("users", userService.allUsers())
        return "display"
    }
}

Web Pages

Finally we have our web pages. One page allows us to register a user, and the other one shows all of our users.

register.html

registercode

display.html

displaycode

Conclusion

Here are some screenshots of what the working site looks like when finished.


As you can see, Spring Security works fluently with ORM solutions such as Hibernate. This makes it much easier to add and retreive users in a web application!

You can clone the full source for this project from my GitHub page here or view the YouTube view here.

Node.js Rest Service Calls

In many instances, an application may need to make a call to a REST service to retrieve relevant information. The Node.js request package offers support for this sort of task.

Begin by creating a Node.js application using Handlebars as the template engine. You can either use your IDE or follow this tutorial to do this from the command line.

Once you have created a skeleton project, head over to OMDB to request an API key. You will get an email that will contain an API key. Then type npm install request to install the request package. Once you have the email, create a keys.js file in the routes folder.

keys.js

The keys.js file is used to hold API keys. Since API keys are like passwords, it’s generally a bad idea to push them to a public repository. By placing them in a file, you can import the keys into a project and then add them to .gitignore to keep them save. Here is what the keys.js file should look like.

exports.omdb_api = 'Your API Key';

index.js

The index.js is used for HTTP GET and POST requests to our application. You can view this tutorial to get an idea of how MVC works in Node.js with Express. We are going to mainly focus on the POST portion of this file.

var express = require('express');
var router = express.Router();
var key = require('./keys.js').omdb_api;
var request = require('request');

var movie = [];

/* GET home page. */
router.get('/', function (req, res) {
    res.render('index', { title: 'Rest Example', movie: movie });
});

router.post('/', function(req, res){
    var query = req.body.query;
    var url = 'http://www.omdbapi.com/?apikey=' + key + '&t=' + query + '&y=&plot=short&r=json';

    //Clear out movie
    movie = [];

    request(url, function(error, response, body){
        //Check for HTTP Status OK
        if (response.statusCode === 200){
            //Convert the body to a JSON object
            var json = JSON.parse(body);

            //Check if it has an error
            if(json.Error){
                movie = json.Error;
            } else {
                //Otherwise, add our movie information to movie
                movie.push({
                    title: json.Title,
                    year: json.Year,
                    imdb: json.Ratings[0].Value,
                    tomatoes: json.Ratings[1].Value,
                    country: json.Country,
                    plot: json.Plot,
                    actors: json.Actors
                });
            }
        } else {
            //We had something other than HTTP OK
            //Push an error to movie and just pass the body
            movie.push({Error: body});
        }
        //Render the index page
        res.render('index', {title: 'Rest Example', movie: movie[0]});
    });
});

module.exports = router;

We begin on line 3, where we import our keys.js file and grab it’s omdb_api variable. This variable holds our API key and will be used to create our URL for our web request. On line 6, we create a movie variable and initialize it to an empty array.

Our POST handler is located on line 13. One line 14, we grab the name of the movie the user wishes to inspect from the req.body.query variable. On line 15, we assemble our url by adding our API key and movie name to the url string.

Line 20 uses the request package to make a call to the OMBD API. It takes two parameters, a url and a call back function. The callback function can have 3 parameters: error, response, and body. We are interested in response and body in this case. Our first job is to check the HTTP status code from response.statusCode. If everything is OK, the response will be 200 (for HTTP OK). Assuming all went well, we can convert the body variable into a JSON object so that we can access the properties of the response.

If the user happens to enter a move that doesn’t exist, the json will have an Error property. We will just assign this to the movie variable is that’s the case. Otherwise, we can create a new object containing title, year, imdb, tomatoes, country, plot, and actors (lines 31-38) and push it to movies. Finally we can return the reponse body back to the view and render the index page (line 47).

layout.hbs

We need to add our Jquery and Bootstrap libraries to our layout.hbs file so that they are available to our pages.
layout

index.hbs

This page renders the results our request.
index.js
The main take away is that we use the {{#if [value]}} markup so that the template engine can decide if it wants to render the error or movie information.

Conclusion

When everything is complete you will get a site that looks like the one shown in the screenshots below.


You can view the full source at my GitHub page here.

Create Node.jS & Handlebars & Express.js Project from command line

It’s trivial to create an empty Node.js project using express generator using the command line. You can follow the guide found at this link for the official express generator example or just keep reading.

Begin by opening up your terminal and navigate to any folder on your file system. Keep in mind that the project will be created as a sub-directory of the folder you choose. If you haven’t installed express generator then execute the following command.

npm install express-generator -g

This command will install express generator on your system. Once the installation is complete, you can create an express project with the handlebars view engine using this command.

express --hbs [application name]

Replace [application name] with the name of your application. For example express -hbs myhbsproject. The script will create a folder in the current directory that has the same name as your application. You should see terminal output that looks similar to the screen shot.
create_the_project
If you look closely, the script tells you the next two steps. First you need to install the dependencies.

cd [application name]
npm install

Once again, npm will do the job of downloading all required project dependencies.
install_packages
Finally, you can run the project by issuing the next command in the terminal.

DEBUG=[application name]:* npm start

You will see this output in the terminal.
run_server
When the server is running, you can open up your web browser and point it at http://localhost:3000/ to see the default homepage.
homepage

Folder Structure

For those people who are curious, you will get the following folder structure when using express generator to create a project.
structure
The public folder is for files such as client side javascript, css, and images that can be referenced by a web page. You should put your server js in the routes folder. All handlebar template files should get placed in the views folder.

You can view the YouTube video here.

Node.js & Spotify

Spotify provides APIs that allow developers to write client applications. This tutorial will demonstrate how to use Node.js to create a simple web application that queries Spotify for information about a particular song. Start by creating a Node.js application with a folder structure that resembles the one shown in this screenshot. You can view a tutorial one how to do this at this link or use your IDE.
Folder_Structure You will want to have the following dependencies in your package.json file.

{
  "name": "spotifynode",
  "version": "0.0.0",
  "private": true,
  "scripts": {
    "start": "node ./bin/www"
  },
  "dependencies": {
    "body-parser": "~1.17.1",
    "cookie-parser": "~1.4.3",
    "debug": "~2.6.3",
    "express": "~4.15.2",
    "hbs": "~4.0.1",
    "morgan": "~1.8.1",
    "node-spotify-api": "^1.0.5",
    "serve-favicon": "~2.4.2"
  }
}

Spotify requires developers to create application keys in order to use their APIs. Follow the guide provided here in order to create a developer account with Spotify. Once you have created an application, you will need to retain the application id and secret.

Start by creating a keys.js file in the routes folder. It should look like the following example.

keys.js

exports.spotifyKeys = {
    id: 'Spotify Id Here',
    secret: 'Spotify Secret Here'
};

The next thing to do is to write the server side code that handles HTTP GET and POST requests. Here is the code for index.js.

index.js

var express = require('express');
var router = express.Router();

//Import the Spotify API
var Spotify = require('node-spotify-api');

//Import our Keys File
var keys = require('./keys');

//Create a Spotify Client
var spotify = new Spotify(keys.spotifyKeys);

//Store the results of a request to spotify
var results = [];

/* GET home page. */
router.get('/', function (req, res) {
    res.render('index', {title: 'Spotify', results: results});
});

router.post('/', function (req, res) {
    //Get the type of Query from the User
    var type = req.body.param_type;

    //Get the query from the user
    var query = req.body.param_query;

    //Clear out old results
    results = [];

    //Make a request to Spotify
    spotify.search({type: type, query: query})
        .then(function (spotRes) {

            //Store the artist, song, preview link, and album in the results array
            spotRes.tracks.items.forEach(function(ea){
                results.push({artist: ea.artists[0].name,
                              song: ea.name,
                              preview: ea.external_urls.spotify,
                              album: ea.album.name});
            });
            //Render the homepage and return results to the view
            res.render('index', {title: 'Spotify', results: results});
        })
        .catch(function (err) {
            console.log(err);
            throw err;
        });
});

module.exports = router;

This code sets up two handlers for GET and POST requests. More details about how to do this can be found in this post. We begin on line 5 by importing the Spotify API into our script. Then we pull in the keys.js file we created earlier so that we can authenticate with Spotify. The next line creates a spotify object and we pass our creditionals to its constructor.

The next point of interest is the spotify.search found on line 32. The spotify.search function takes in two arguments, type and query. The type argument specifies the type of query and the query is the actual search criteria that we are going to send to the API. The spotify library will make correct rest calls to the Spotify API and it will return a response.

Inside of the body of the promise function, we push some information about the song to the results array so that we can display it to the view. In this case, we are going to grab the artist, song, a preview url, and the song’s album. We then return it to the view for display in a table.

index.hbs

We can use Handlebars to markup a template page that will get returned to the browser from the server.
spotify-index

Conclusion

When run, the application will look like the following screenshots.


You can view the source from my GitHub page at this link.

Node.js Handlebars Twitter

Twitter provides REST APIS that make it incredibly easy to access information from Twitter. When combined with Node.js, it becomes a trivial task to create a web application that connects to Twitter and displays the latest tweets for a User.

In order to begin, you will need to register at Twitter’s developers page. The details page will give you several tokens that you will need to connect to Twitter: consumer_key, consumer_secret, access_token_key, and access_token_secret. Keep this information handy as you will need it for later.

Now we can create the project. I use Intellij’s node.js plugin to create my project, but you will want to create a project structure that is similar to the one shown in the following screenshot. You can view a tutorial one how to do this at this link or use your IDE or you may find the directions at Express.js to be useful as well.
project_structure
Once you have created your project (making sure you have express.js and handlebars.js), you will need to add Twitter. Open up the terminal and navigate to your project’s directory. Then type

npm install twitter

At this point you are ready to begin developing.

Create a keys.js file in your routes folder. Then populate it with the following code. You will use the consumer_key, consumer_secret, access_token_key, and access_token_secret properties that you obtained earlier on Twitter’s developer’s page and replace the values in this file accordinly.

exports.twitterKeys = {
    consumer_key: '[your consumer key here]',
    consumer_secret: '[your consumer secret here]',
    access_token_key: '[your access_token_key here]',
    access_token_secret: '[your acess_token_secret here]',
};

The next job is to write the index.js file that handles HTTP GET and POST requests. (You can view this tutorial for more information). The code is very short as we are only handling a GET and POST request.

//Pull in our libraries
var express = require('express');
var router = express.Router();
var Twitter = require('twitter');

//Get our access keys
var keys = require('./keys.js');

//Create twitter client
var twitter = new Twitter(keys.twitterKeys);

//Store our tweets
var tweets = [];

/* GET home page. */
router.get('/', function (req, res) {
    res.render('index', {title: 'Node Twitter', tweets: tweets});
});

router.post('/', function (req, res) {
    //Get the user name and number of tweets from the form
    var user = req.body.user_name;
    var numTweets = req.body.tweets;

    //Clear out old tweets
    tweets = [];

    //Hit Twitter for the information
    twitter.get('statuses/user_timeline', {screen_name: user, count: numTweets})
        .then(function (tw) {
            //Loop through the results
            for (var i = 0; i < tw.length; i++) {
                tweets.push({timestamp: tw[i]['created_at'], tweet: tw[i]['text']});
            }
            //Render the index page with the tweets
            res.render('index', {title: 'Node Twitter', tweets: tweets});
        })
        .catch(function (error) {
            console.log(error);
            throw error;
        });
});

module.exports = router;

The magic of retrieving tweets is done with the twitter.get(). This function takes a rest endpoint (see here for a list) along with a map of arguments that end up becoming URL parameters in the request. Since we are getting the user’s tweets, we are going to hit the statuses/user_timeline endpoint. The screen_name argument is the users Twitter name and count is the number of tweets that are getting returned.

The Twitter api returns a massive JSON response that holds just about every detail we could ever want to know about a Tweet. We are only interested in when the tweet was created and the text of the tweet. So we loop through the tw (the response object) and just grab those two properties and push them into our tweets array. When we have finished, we will pass tweets back to the view and render the index page.

Our final job is to define our html templates. Since we are using Bootstrap and jQuery, we will need to add these libraries to layout.hbs.
layout.hbs
Next we need to write our index.hbs template. Here is the source.
index
Line 25 starts with {{#if tweets}}. This is a conditional rendering tag that only renders the code between 25-42 if there are tweets to display. Without it, we would write out the table between lines 26-41 when there are no tweets, which would not make a lot of sense.

Once we have tweets, the #if condition will be true and the page will write out a table of tweets. The code on line 34-39 will write out a table row for each tweet that is contained. When we are finished, we will get this output.

You can get the code out my GitHub page here or view the video tutorial on my YouTube channel.

Kotlin Spring JDBC Template

It’s typical for many applications, including web applications, to read and write to a database. JDBC operations are significantly simplified when using Spring JdbcTemplates and Kotlin’s language features. For example, it’s easy to one line read and insert operations into a database. This post goes through a sample web application that inserts a user into a database table and then prints a list of all users stored in the database.

Interacting with the Database

Our first order of business is to create a database schema. This is the SQL script that we will use to generate our database.

DROP TABLE IF EXISTS USERS;

CREATE TABLE USERS (
  id INTEGER IDENTITY,
  first_name VARCHAR(50),
  last_name VARCHAR(50),
  email VARCHAR(50),
  phone VARCHAR(50),
)

Now we will define a database that maps to the information held in our database. Kotlin’s data classes are ideal for this sort of task.

//Define a data class that maps to both our
//form and database table
data class User(var firstName: String = "",
                var lastName: String = "",
                var email: String = "",
                var phone: String = "")

There isn’t anything special about our User class. It’s only job is to carry information from the view to the database and back from the database to the view. Now that we have a database table and a transfer object, we need to configure our datasource so that Spring can connect our application to our database. We will define a configuration class that will define some Spring beans for us.

@Configuration
class Configuration {

    //First configure a data source that
    //generates an embedded db
    @Bean(name = arrayOf("dataSource"))
    fun dataSource(): DataSource {

        //This will create a new embedded database and run the schema.sql script
        return EmbeddedDatabaseBuilder()
                .setType(EmbeddedDatabaseType.HSQL)
                .addScript("schema.sql")
                .build()
    }

    //Create a JdbcTemplate Bean that connects to our database
    @Bean
    fun jdbcTemplate(@Qualifier("dataSource") dataSource: DataSource): JdbcTemplate {
        return JdbcTemplate(dataSource)
    }
}

The first bean, dataSource, returns an EmbeddedDatabaseBuilder object that does the work of creating an embedded database, setting it’s dialect, and running our schema.sql script to create the database definition. At this point, our database is fully ready when the build() method is called.

The next bean is a JdbcTemplate object. We create a bean definition for it so that we can inject instances of this object into our repository classes later on. The JdbcTemplate requires a DataSource object, which happens to point at our embedded database. Now let’s define a repository class that will actually work with our JdbcTemplate.

@Repository
class IndexRepository(@Autowired var jdbcTemplate: JdbcTemplate) {


    fun addUser(user: User) {
        //We can use SimpleJdbcInsert to insert a value into our table
        //The becomes super concise when combined with Kotlins apply and mapOf functions
        SimpleJdbcInsert(jdbcTemplate).withTableName("USERS").apply {
            setGeneratedKeyName("id")
            execute(
                    mapOf("first_name" to user.firstName,
                            "last_name" to user.lastName,
                            "email" to user.email,
                            "phone" to user.phone))
        }
    }

    //This allows us to query the Users table and return a list of users
    //This is one method call to jdbcTemplate with a lambda expression which makes the code
    //incredibly concise
    fun allUsers(): List = jdbcTemplate.query("SELECT FIRST_NAME, LAST_NAME, EMAIL, PHONE FROM USERS",
            { rs: ResultSet, _: Int ->
                User(rs.getString("first_name"), rs.getString("last_name"), rs.getString("email"), rs.getString("phone"))
            })
}

@Respository is a Spring sterotype annotation that marks our IndexRepository as a class that is intended to interact with the datasource. Spring provides two other stereotype annotations, @Controller and @Service, that are typically used to mark seperations in the application. @Controller is intended to interact with the view, while @Respository works with datasource. @Service should contain business logic. When developers follow this pattern, the application maintains loose coupling which makes it easy to maintain and test code.

Since IndexController is marked with @Repository, it makes sense to inject JdbcTemplate into this class so that it can work with the database. We have two methods in this class: addUser and allUsers. We’ll take each function on its own.

The addUser(user : User) method performs an insert into the database. We create an instance of SimpleJdbcInsert and pass our JdbcTemplate object into this class. The following call to withTable(“USERS”) specifies which table we are inserting a record into. Since our primary key is generated automatically by the database, we can use SimpleJdbcInsert.setGeneratedKeyName(“id”) to assign a primary key. Finally we use the execute() function to actually perform the insertion into the database. The execute() takes a map where the key is the name of the column in the database and the value is what we are inserting into the column.

There is some Kotlin magic that helps keep the code concise. For one, we are chaining our calls to setGeneratedKeyName() and execute() inside of the apply() function. We can also leverage Kotlin’s mapOf() function to generate a Map on the fly as opposed to creating a map object and populating it with values ahead of time.

The allUsers() function queries the database. In this case, we can call the query method from the jdbcTemplate object. The query() method requires two parameters. The first parameter is the query that is sent to the database. The second method is a an instance of RowMapper, which is a single abstract method (SAM) class. Since RowMapper has only one method, we can use a lambda expression to provide an implementation of RowMapper.

The RowMapper’s job is to transform the results of the database query into a User object. It provides with two objects that help with this job. The first is the good old JDBC ResetSet object and the other object is an Int that represents the row number. We only use the ResultSet in this example. The ResultSet interface has a getString() method that takes the name of the column and outputs the value stored in that column. Using getString(), we can populate each field of a User object and return it. RowMapper will handle the details of building a list and returning the List to the caller.

Web Portion

The remaining part of the application is a Spring MVC application. We aren’t going to spend a lot of time on this portion but are including it for After the @Repository tier (covered above), we have a service class that handles the business logic between the @Controller and the @Repository. In our case, it’s really boring because all our @Service class is doing is acting as a wrapper for our @Repository class, but in the real world, there is generally more application code located in this class.

@Service
class IndexService(
    //Inject IndexRepository here
    @Autowired var indexRepository: IndexRepository) {

    fun addUser(user: User) {
        indexRepository.addUser(user)
    }

    fun allUsers(): List {
        return indexRepository.allUsers()
    }
}

We also have a @Controller class that handles HTTP GET and POST requests.

@Controller
@RequestMapping("/")
class IndexController(@Autowired var indexService: IndexService) {

    @RequestMapping(method = arrayOf(RequestMethod.GET))
    fun doGet(model: Model): String {
        model.addAttribute("user", User())
        model.addAttribute("allUsers", indexService.allUsers())
        return "index"
    }

    @RequestMapping(method = arrayOf(RequestMethod.POST))
    fun doPost(model: Model, user: User): String {
        indexService.addUser(user)

        model.addAttribute("user", User())
        model.addAttribute("allUsers", indexService.allUsers())
        return "index"
    }
}

And finally the view…
View

Conclusion

Kotlin greatly enchances the already excellent JDBC abilities offered by Spring Boot. As was demonstrated in this post, developers can start with definining a data class that holds all of the data in a single row in a database table. When it comes to actually perform inserts or queries from the database, Kotlin’s mapOf(), apply, and to functions cut down on any additional verbosity that might still remain from using JDBC Template. As always, Spring makes it super simple to spring up a web application that interfaces with a database.

You can grab the source for this example from my GitHub page or view the Video tutorial on YouTube.

Use Handlebars and Express in Node.js

Node.js is an excellent server side platform that makes it really easy to write and deploy sites. In this post, I will demonstrate how to use Node.js in combination of Express and Handlebars to create an example registration form. You can view a tutorial one how to create a project at this link or use your IDE.

Views

We need to define a couple of web pages for the view. The first page is a master template that is used when serving every page in the site.

layout.hbs

Layout_hbs

There are a couple of interesting aspects about the layout.hbs code fragment. For one thing, it provides us with a common place to define our site level css and javascript. In this example, both jQuery and Bootstrap are added in this file and are therefore included in every other page.

The other aspect is the {{{body}}}. The layout.hbs code does not have its own body. Instead, it use a placeholder {{{body}}} that represents what will utlimately become the body of each html page. Our next two pages only define the body portion of the page and they are inserted into the html page at the {{{body}}}.

index.hbs

The index.hbs is one of two files that are inserted at the {{{body}}} portion of the labyout.hbs file. This file provides the user with an example registration form.

index_hbs

In all truthfulness, there isn’t anything special about this page whatsoever. This page is literally a regular html form with some bootstrap css to make it look presentable. The magic happens on the backend with the index.js file.

index.js

This file contains the code that handles both HTTP GET and HTTP Post requests.

var express = require('express');
var router = express.Router();

/* GET home page. */
router.get('/', function(req, res, next) {
res.render('index', { title: 'Node Tutorial' });
});

router.post('/', function(req, res){
//Grab the request body
var body = req.body;

var res_body = {
first_name: body.first_name,
last_name: body.last_name,
email: body.email
};

res.render('welcome', res_body);
});

module.exports = router;

This script work by creating two variables. First an express variable followed by a router variable. The router is the object that does the work of handling HTTP requests. The first function is a router.get(‘/’, function(…){…}). That function handles all HTTP GET requests to our index page which is mapped to the root of the site “/”. The call back function takes three parameters: req, res, and next. Next is unused in this example but req stands for Request and contains the request body the browser sent back to the server. The res stands for Response and represents the response the server sends back to the browser.

The function for our GET request has only one line of code. res.render('index', {title: 'Node Tutorial' }). Although it doesn’t seem like much, that single line of code packs a massive punch. For one thing, it is tell the server to combine the layout.hbs and index.hbs into one html document by inserting index.hbs at the {{{body}}} placeholder in layout. It’s also telling the template engine to insert the phrase ‘Node Tutorial’ at the {{title}} placeholder found in layout.hbs. When these operations are complete, the browser gets an html file that resembles the screen shot below.

homepage

The next function is the router.post('/', function(...){...}). This function handles http post requests for the homepage. On the first line of this function, we store the request body inside of a body variable. Each property on the body, first_name, last_name, email, pw, pw_confirm all represent our text boxes in index.hbm as specified by the name attribute on each of those input fields. So when the user enters ‘Bob’ in the box labeled as First Name on our form, the body.first_name gets set to the String ‘Bob’. This is true for each field on the form.

All we are doing in this particular example is gathering the first_name, last_name, and email fields from the request body and storing them in an object res_body. The last line of code res.render('welcome', res_body); tells the server to redirect to the welcome.hbm page and pass the res_body variable to the template engine. This will allow the template engine to access the values stored in res_body when it renders the the welcome page.

welcome.hbs

The final page of this project is the welcome.hbm page. This page is incredibly similar to the index.hbs page in that it gets inserted into layout.hbs at the {{{body}}} variable. Here is the code for welcome.hbm

welcome_hbs.png

This page defines three placeholders: {{first_name}}, {{last_name}}, and {{email}}. All of these place holders map to the properties in the res_body variable that we sent back to the view in index.js. When the template engine sees the res_body variable, it knows to insert res_body.first_name at {{first_name}}, res_body.last_name at {{last_name}}, and res_body.email at {{email}}. Below is a screenshot of the result.

Conclusion

Node.js make it really easy to create an MVC style site. Developers need to leverage the router variable and map html end points to various HTTP request and then respond as needed. Information can be easily sent back to the view using JavaScript objects where the values can get inserted into Handlebars templates.

You can get the code for this tutorial at my Github page.
Visit the video tutorial at https://youtu.be/rEdQ6fvSNOk