Cake Build Tool

I don’t know exactly when or where I first came across the Cake build tool, and at the time I made a mental note to look at it in more detail (as I am not a massive fan of MSBuild). That time came and went, and I did nothing about it. Then Cake came across my radar again so this time I decided to dig into it a bit more.

 

So what is this cake build tool?

The Cake build tool is a build tool that utilizes the Roslyn (compiler as a service) from .NET. What this means is that you can write very precise build scripts using very familiar C# language syntax that you know and love.

 

Getting started

The best way to get started is to clone the example repo : https://github.com/cake-build/example

The repo is a simple C# class library and a test project all within a single solution.

 

image

As you can see this project is very simple. What we would like to do with this project is the following things :

  • Clean solution
  • Restore Nugets
  • Build solution
  • Run tests
  • And also have ability to push out Nuget package (nupkg file)

Most of this is already available within the example repo : https://github.com/cake-build/example, with the exception of pushing a nuget package at the end.

 

What bits do you need to run a cake build?

So what do you need to provide to run a cake build

You just need these 2 files

  • build.ps1 (bootstrapper that doesn’t change, grab it from repo example above)
  • build. cake (this is your specific build and should contain the targets/tasks you need for your build)

 

The .cake file

As the build.ps1 is a standard thing I won’t worry about that, but lets now turn our attention to the build.cake file which for this post looks like this

 

#tool nuget:?package=NUnit.ConsoleRunner&version=3.4.0


//////////////////////////////////////////////////////////////////////
// ARGUMENTS
//////////////////////////////////////////////////////////////////////

var target = Argument("target", "Default");
var configuration = Argument("configuration", "Release");

//////////////////////////////////////////////////////////////////////
// PREPARATION
//////////////////////////////////////////////////////////////////////

// Define directories.
var buildDir = Directory("./src/Example/bin") + Directory(configuration);

//////////////////////////////////////////////////////////////////////
// TASKS
//////////////////////////////////////////////////////////////////////

Task("Clean")
    .Does(() =>
{
    CleanDirectory(buildDir);
});

Task("Restore-NuGet-Packages")
    .IsDependentOn("Clean")
    .Does(() =>
{
    NuGetRestore("./src/Example.sln");
});

Task("Build")
    .IsDependentOn("Restore-NuGet-Packages")
    .Does(() =>
{
    if(IsRunningOnWindows())
    {
      // Use MSBuild
      MSBuild("./src/Example.sln", settings =>
        settings.SetConfiguration(configuration));
    }
    else
    {
      // Use XBuild
      XBuild("./src/Example.sln", settings =>
        settings.SetConfiguration(configuration));
    }
});

Task("Run-Unit-Tests")
    .IsDependentOn("Build")
    .Does(() =>
{
    NUnit3("./src/**/bin/" + configuration + "/*.Tests.dll", new NUnit3Settings {
        NoResults = true
        });
});


var nugetPackageDir = Directory("./artifacts");
var nuGetPackSettings = new NuGetPackSettings
{   
  OutputDirectory = nugetPackageDir  
};

Task("Package")
  .Does(() => NuGetPack("./src/Example/Example.nuspec", nuGetPackSettings));


//////////////////////////////////////////////////////////////////////
// TASK TARGETS
//////////////////////////////////////////////////////////////////////

Task("Default")
    .IsDependentOn("Run-Unit-Tests");

//////////////////////////////////////////////////////////////////////
// EXECUTION
//////////////////////////////////////////////////////////////////////

RunTarget(target);

 

 

There are a couple of concepts to call out there

 

  • We have some top level arguments/ variables
  • Nice C# features that we have used before
  • We have Tasks just like other build systems. We can make one task depend on another using .IsDependantOn(“”)
  • There seems to be wide range of inbuilt things we can use for example these guys below. These are all prebuilt items in the cake DSL that we can make use of. There are loads of these, the full list is available here : https://cakebuild.net/dsl/
    • CleanDirectory
    • NUnit3
    • NuGetPack

 

Have a look at the DSL web site there are quite a few cool things you can use

 

image

 

Running the build

So with this build.cake and build.ps1 (bootstrapper file) in place we would like to run the build. Here is how we do that

 

1. Open PowerShell window as Administrator
2. Issue this command in PowerShell : Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
3. Change to the correct directory with the .cake file in it, and issue this command : .\build.ps1
4. You should see some output, where it eventually completes
5. You should also see a tools folder

 

This is the tail end of the build I just ran above

 

image

 

And this is the sort of thing that we should see in the tools folder that the cake build created

 

image

 

Deploying a Nuget

So I stated that I also wanted to be able to deploy a Nuget Package as a Nupkg. To do this I need to create the following .nuspec file for the Example project

<?xml version="1.0"?>
<package >
  <metadata>
    <id>Example</id>
    <version>1.0.0</version>
    <title>Cake Example</title>
    <authors>Sacha Barber</authors>
    <owners>Sacha Barber</owners>
    <licenseUrl>http://github.com/sachabarber</licenseUrl>
    <projectUrl>http://github.com/sachabarber</projectUrl>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <description>Simple Cake Build Tool Example</description>
    <releaseNotes>1st and only release</releaseNotes>
    <copyright>Copyright 2018</copyright>
    <tags>C# Cake</tags>
  </metadata>
  <files>  
   <file src="bin\Release\Example.dll" target="lib\net45"></file>  
</files> 
</package>

 

So with that in place we can also try the Nuget publish Task that our build.cake file has in it like this:

 

1. Open PowerShell window as Administrator
2. Issue this command in PowerShell : Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
3. Issue this command in PowerShell : .\build.ps1 -Target Package

 

After running that we should see artifacts folder with the following artifact in it

 

image

 

Conclusion

I was pretty happy with this, I went from not using Cake at all to carrying out ALL my requirements in 1 hour on a train ride with limited WiFi. It just seems to work, and I imagine it would be a good fit for working with something like https://about.gitlab.com/

 

I think I will be looking to use this little build tool a lot more.

Advertisements

MADCAP IDEA PART 4 : PROTOTYPING THE SCREENS

 

Last Time

 

Last time we looked at bringing in the Play Framework (scala based MVC web framework) and making the front end work with Play. This time we will  be look at the initial prototypes of the screens.

This is my best guess of what they may look like right now, based on my initial requirements, but as with all things once you get into the guts of it, changes will occur.

 

 

PreAmble

Just as a reminder this is part of my ongoing set of posts which I talk about here :

https://sachabarbs.wordpress.com/2017/05/01/madcap-idea/, where we will be building up to a point where we have a full app using lots of different stuff, such as these

  • WebPack
  • React.js
  • React Router
  • TypeScript
  • Babel.js
  • Akka
  • Scala
  • Play (Scala Http Stack)
  • MySql
  • SBT
  • Kafka
  • Kafka Streams

 

Mockup tool of choice

I am a big fan of the balsamiq mockup tools https://balsamiq.com/. This comes as a stand alone installed version or as a plugin for JIRA.

balsamiq provides the following (I am just listing the features I used, there are many more)

 

  • Drag and drop from a wide range of forms, containers, controls
  • Set content for controls (usually using some fancy design time behavior)
  • Set navigation links
  • Set properties like IsSelected, IsEnabled etc etc

 

This is what the windows installed balsamiq desktop version looks like, see how you have many categories of items to choose from

 

image

 

And here is what I mean by the clever design time support. This is a data grid that I have double clicked on, where the text in design mode described the rendered results of the control

 

image

 

It really is a very nice tool. Anyway on with the initial screen designs

 

Navbar

image

 

This will be a simple react-router / react-bootstrap based navigation bar. There is nothing much more to say about that.

 

 

Login

image

 

This will be a login form which will be validated, and submitted to a Play framework controller, for further validation. The Play controller would look up the user details from a MySQL database, and if an entry is found the user is considered logged on. Keeping in simple here no oAuth no JWT, just simple lookup

 

Passenger Register

image

 

If the user is a passenger the sort of information that they will need to enter to register will be different from a driver who may need to register, as such there is a specific passenger registration form, which will be validated and sent to a Play controller endpoint for storage in MySQL.

 

Driver Register

image

 

If the user is a driver, we need more information about the vehicle, as such there is a specific driver registration form, which will be validated and sent to a Play controller endpoint for storage in MySQL.

 

 

Create Job

image

 

Only a passenger will be able to create new jobs. Since I am doing all this work on a single laptop which is ALWAYS in a single location, I am having to SIMULATE the geo-coordinates of a job by accepting the current users input for their current position. The passenger/driver users will provide this geo information by clicking on a google map. The geo co-ordinate update will either travel through a Kafka stream, –> Akka –> Comet, or may just use Akka –> Comet. I have not fully decided on this part yet.

 

There may only EVER be 1 active job, so if a logged in passenger tries to create a 2nd job this should cause an error

 

image

 

View Job

Both passengers/drivers may view an active job. Drivers may “bid for a job” by clicking on the map providing the job is not already paired with a driver.  A driver symbol will be a car, as before the driver will update their geo co-ordinates by clicking on the map. A before the geo co-ordinate update will either travel through a Kafka stream, –> Akka –> Comet, or may just use Akka –> Comet.

 

image

 

A passenger may inspect a drivers details, and chose to accept the driver, at which point the passengers job become assigned the chosen driver.

 

image

 

Drivers that are not allocated to the job will be removed from the map, and only geo updates from the paired passenger/driver will be reflected on the map. 

 

Passenger Completion

 

image

Once a job has been completed (by clicking the”Complete” button) the passenger will be able to rank the driver. This will store the ranking for the driver. This could be stored directly in MySQL, but I want to play with Kafka Streams a bit more, so we use a Kafka Publisher –> Kafka Streams –> KTable arrangement to store the state. And then use Kafka active queries to get the data out again.

 

 

Driver Completion

image

 

The driver is also able to complete the job from their end (using the “complete” button), and is able to rank the passenger. This will work as described above.

 

View Ranking

image

 

Depending on which way I go with the ranking storage this will either be a direct MySQL query or a Kafka Streams active query over a KTable.

 

Conclusion

This is perhaps the simplest of all the posts in this series, but it is an important one. Next time we will try and statically implement these screens, and the associated routing that goes with them.

MADCAP IDEA

INTRODUCTION

So this is somewhat of strange post, or should I say what will hopefully become a decent set of posts, thing is, I have no idea how this will end up really,
as I have not embarked on a mission like this before. So please bear with me.

SO JUST WHAT IS IT THAT I AM TALKING ABOUT?

Well the way I typically like to run my blog/code project articles / life, is that I pick a technology and
just concentrate on it for a while and write about it. This time however I have decided to treat my blogging/articles as a bit more
of a work like escapade, where I will be assigning mini tasks (think JIRA tickets) to myself, some of which I know nothing about, that should/could in reality be treated
as “spikes” and end up in complete dead ends. It is about the journey after all.

I WILL have a complete list of “tickets” (AKA tasks), which may or may not be completely fleshed out in advance. I will stick to “DOING” those “tickets”
and there is an end goal in sight, and I will outline that in a top level story. I cannot however commit to any timelines, this is as much my journey as it is yours (in fact I mainly
do this stuff for myself, and would highly reccomend it as a way of self improvement). That said I hope people get something out of the series of posts that WILL UNDOUBTEDLY
come from this idea.

You can think of the tasks as “technical tasks” which make up the high level “stories” (in JIRA speak).

This may come across a bit weird, but the technogies I plan to cover in the final product is pretty much a full app, so it’s a little hard to describe in
one blog post/article. So I am hoping that by breaking it down into small chunks, each story/sub task will be a useful learning experience in
it’s own right.

SOURCE CONTROL : ORGANISATION

The idea is that each story/sub task will be a folder/subfolder which is completely independent of other stories/sub tasks (up until the final goal, which is of course
a working showcase that demostrates it all working together).

NOTE TO SELF : I am going to try really hard to do this (aren’t we sacha), as I think one topic -> one source control repo (more than likely GIT), is a good way to
correlate ideas/words on the post/article

 

WHAT DO I WANT TO WRITE


In essence I want to write a very (pardon the pun) but uber simple “uber” type app. Where there are the following funtional requirements

  • There should be a web interface that a client can use. Clients may be a “driver” or a “pickup client” requireing a delivery
  • There should be a web interface that a “pickup client” can use, that shows a “pickup client” location on a map, which the “pickup client” choses.
    The “pickup client” may request a pickup job, in which case “drivers” that are in the area bid for a job.
    The “pickup client” location should be visible to a “driver” on a map
  • A “driver” may bid for a “pickup client” job, and the bidding “driver(s)” location should be visible to the “pickup client”.
  • The acceptance of the bidding “driver” is down to the “pickup client”
  • Once a “pickup client” accepts a “driver” ONLY the assigned “driver(s)” current map position will be shown to the “pickup client”
  • When a “pickup client” is happy that they have been picked up by a “driver”, the “pickup client” may rate the driver from 1-10, and the “driver” may also rate the “pickup client” from 1-10.
  • The rating should only be available once a “pickup client” has marked a job as “completed”
  • A “driver” or a “pickup client” should ALWAYS be able to view their previous ratings. 

Whilst this may sound child’s play to a lot of you (me included if I stuck to using simply CRUD operations), I just want to point out that this app is meant as a learning experience so I will not be using a simple SignalR Hub, and a couple of database tables.

I intend to write this project using a completely different set of technologies from the norm. Some of the technology choices could easily scale to hundreds of thousands of requests per second (Kafka has your back here)

POTENTIAL TECNHOLOGIES INVOLVED

  • WebPack
  • React.js
  • React Router
  • TypeScript
  • Babel.js
  • Akka
  • Scala
  • Play (Scala Http Stack)
  • MySql
  • SBT
  • Kafka
  • Kafka Streams

Some of this will undoubtedly be covered in other blogs (such as React/Webpack), however some of it I am hoping will be quite novel/insightful material.

Who knows though there may be some of you out there that haven’t heard of Webpack, so some of that may even be new, we shall se, hopefully enough stuff for everyone.

STORIES

I will maintain a list of stories and their sub tasks using Trello here : https://trello.com/b/F4ykCOOM/kafka-play-akka-react-webpack-tasks which at the time of writing this post was the items shown below

 

TOP LEVEL STORIES

Web Site

Play Back End

Kafka Streams

Create test app that tests out listening to any single Kafka publisher JSON topic, and creates streams app from it, and pushes out to an output topic

 

 

HOW WILL PROGRESS BE TRACKED

I will simply use Trellos “Label” facility, such that done tasks will be “Green”, and there will obviously be a post/GitHub code repo folder that goes with that.

 

CAVEATS

1. I will not be concerned with connection failures, the aim of the project is to try and create a real world like project, but not actually create a end-end production grade application
2. I will be treating every run as if it were the first, I will not be storing ANY permanent state (apart from ratings potentially)
3. I will be doing things at my own pace (I have 2 kids) so it comes when it comes

4. I will try and use varied technology choices, which will in places mean that there could potentially be more work required to make it production quality

 

 

 

A Look At Docker

A while ago I worked on a project that used this tech stack

  • Akka HTTP : (actually we used Spray.IO but it is practically the same thing for the purpose of this article). For those that don’t know what Akka HTTP is, it is a simple Akka based framework that is also able to expose a REST interface to communicate with the actor system
  • Cassandra database : Apache Cassandra is a free and open-source distributed database management system designed to handle large amounts of data across many commodity servers, providing high availability with no single point of failure. Cassandra offers robust support for clusters spanning multiple datacenters, with asynchronous masterless replication allowing low latency operations for all clients.

It is a multi node cluster

This was a pain to test, and we were always stepping on each others toes, as you can imagine running up a 5 node cluster of VMs just to satisfy my each developers own testing needs was a bit much. So we ended up with some dedicated test environments, running 5 Cassandra nodes. These was still a PITA to be honest.

This got me thinking perhaps I could use Docker to help me out here, perhaps I could run Cassandra in a Docker container, hell perhaps I could even run my own code that uses Cassandra in a Docker container, and just point my UI at the Akka HTTP REST server running in Docker. mmmmm

I started to dig around, and of course this is entirely possible (otherwise I would not be writing this article now would I).

This is certainly not a new thing here for Codeproject, there are numerous Docker articles,  but I never found one that talked about Cassandra, so I thought why not write another one.

 

Which I have just published here : https://www.codeproject.com/Articles/1175248/A-look-at-Docker

akka mailboxes

This post will be a bit smaller than the ones we have just done, but none the less still just as important.

Let’ start by talking about mailboxes and dispatchers, and what exactly they are and how they relate to each other.

What’s A Mailbox?

In Akka the mailbox is some time of queue that holds the messages for an actor. There is usually a mailbox per actor. Though in some cases, where routing gets involved there may only be one mailbox between a number of actors, but for now lets assume a one to one relationship between mailboxes and actors.

What’s A Dispatcher

In Akka a Dispatcher is the heart of the actor system, and it is the thing that dispatches the messages.

There is a way that Akka actors may be configured to use a certain Dispatcher and the Dispatcher may in turn be configured to use a certain mailbox type.

Here is an example of how you might configure an actor to use a custom Dispatcher

You may have this code for an actor

import akka.actor.Props
val myActor =
  context.actorOf(Props[MyActor].withDispatcher("my-dispatcher"), "myactor1")

Where you may have this custom Dispatcher in your configuration of the system

my-dispatcher {

  # Type of mailbox to use for the Dispatcher
  mailbox-requirement = org.example.MyInterface

  # Dispatcher is the name of the event-based dispatcher
  type = Dispatcher

  # What kind of ExecutionService to use
  executor = "thread-pool-executor"

  # Configuration for the thread pool
  thread-pool-executor {

    # minimum number of threads to cap factor-based core number to
    core-pool-size-min = 2

    # No of core threads ... ceil(available processors * factor)
    core-pool-size-factor = 2.0

    # maximum number of threads to cap factor-based number to
    core-pool-size-max = 10
  }

  # Throughput defines the maximum number of messages to be
  # processed per actor before the thread jumps to the next actor.
  # Set to 1 for as fair as possible.
  throughput = 100
}

It can be seen above that we are able to configure the mailbox type for a Dispatcher in the configuration using the line

# Type of mailbox to use for the Dispatcher
mailbox-requirement = org.example.MyInterface

There are actually several inbuilt Dispatcher types that you may use when creating a custom Dispatcher.

Talking about Dispatch types and how they all work is kind of out of scope for what I wanted to talk about in this post though, so if you want to know more about Akka Dispatchers you should consult the official Akka documentation

http://doc.akka.io/docs/akka/snapshot/scala/dispatchers.html

Ok so now that we have taken that slight detour and talked about how you can associate a mailbox type with a custom Dispatcher should you want to let’s get back to the main thrust of this post, which is to talk about mailboxes.

As previously stated mailboxes represent a storage mechanism for an actors messages.

Built In Mailbox Types

Akka comes shipped with a number of mailbox implementations:

UnboundedMailbox – The default mailbox

  • Backed by a java.util.concurrent.ConcurrentLinkedQueue
  • Blocking: No
  • Bounded: No
  • Configuration name: “unbounded” or “akka.dispatch.UnboundedMailbox”

SingleConsumerOnlyUnboundedMailbox

  • Backed by a very efficient Multiple Producer Single Consumer queue, cannot be used with
  • BalancingDispatcher
  • Blocking: No
  • Bounded: No
  • Configuration name: “akka.dispatch.SingleConsumerOnlyUnboundedMailbox”

BoundedMailbox

  • Backed by a java.util.concurrent.LinkedBlockingQueue
  • Blocking: Yes
  • Bounded: Yes
  • Configuration name: “bounded” or “akka.dispatch.BoundedMailbox”

UnboundedPriorityMailbox

  • Backed by a java.util.concurrent.PriorityBlockingQueue
  • Blocking: Yes
  • Bounded: No
  • Configuration name: “akka.dispatch.UnboundedPriorityMailbox”

BoundedPriorityMailbox

  • Backed by a java.util.PriorityBlockingQueue wrapped in an akka.util.BoundedBlockingQueue
  • Blocking: Yes
  • Bounded: Yes
  • Configuration name: “akka.dispatch.BoundedPriorityMailbox”

 

Default Mailbox

As shown above the unbounded mailbox is the default. You can however swap out the default using the following configuration, though you will need to ensure that the chosen default mailbox is the correct one for the type of Dispatcher used. For example a SingleConsumerOnlyUnboundedMailbox can not be used with a BalancingDispatcher

Anyway this is how you would change the default mailbox in config

akka.actor.default-mailbox {
  mailbox-type = "akka.dispatch.SingleConsumerOnlyUnboundedMailbox"
}

 

Mailbox Type For An Actor

It is possible to associate a particular type of mailbox with a particular type of an actor which can be done by mixing in the RequiresMessageQueue trait

import akka.dispatch.RequiresMessageQueue
import akka.dispatch.BoundedMessageQueueSemantics
 
class SomeActor extends Actor
  with RequiresMessageQueue[BoundedMessageQueueSemantics]

Where you would use the following configuration to configure the mailbox

bounded-mailbox {
  mailbox-type = "akka.dispatch.BoundedMailbox"
  mailbox-capacity = 1000
  mailbox-push-timeout-time = 10s
}
 
akka.actor.mailbox.requirements {
  "akka.dispatch.BoundedMessageQueueSemantics" = bounded-mailbox
}

It is worth noting that this setting could be overwritten by code or by a dispatcher mailbox configuration section

Where Is The Code?

As previously stated all the code for this series will end up in this GitHub repo:

https://github.com/sachabarber/SachaBarber.AkkaExamples

The Nuances of Loading and Unloading Assemblies with AppDomain

I don’t normallly like just pointing out other peoples work, bit this time I have no hesitation at all in doing just that. If you have ever worked with AppDomain(s) in .NET you would have certainly had some fun.

CodeProject Marc Clifton has written a truly great article on AppDomain(s) which you should all read. You can find it here : http://www.codeproject.com/Articles/1091726/The-Nuances-of-Loading-and-Unloading-Assemblies-wi

Nice one Marc