This is "after speech" post about my thoughts on BDD, coders vs developers and some hype.
Yesterday I held a speech at #ITNightSamara where nearly 50 java developers were watching my demo. My goal was to raise a question about the difference between coding and professional (I mean paid) software development and importance of production in the enterprise world. But a record of my talk will be available later, in this post I want to show you my preparations for this talk and interesting moments of this journey.
"In your heee-ad"
First of all - BDD, I looked at it in early 2015 and it was a very interesting concept for me.
I even tried it on one of my R&D projects, where I was alone (not the only developer, but the only person).
And being Three amigos by myself helped a lot.
You better understand what you are trying to do (and R&D is about trying and failing) if you use Gherkin.
And this year my team faced big trouble. There was a lot of miscommunication between architecture, BA and development.
Usage of some BDD principles, where we write Gherkin features first then wait for BA approval and only after it develops new functionality, made our life (in a professional sense) easier.
Why did that happened? Because a lot of problems in software development could be solved by staying on the same page. If you as a developer or SDET\QA understand the behaviour of feature system the same way architecture and BA understand it (and in fact - end user should understand it the same way) - then it's easier to reach the target and don't code "something", but develop software.
Story
So to prepare the demo I started with this story it follows BDD principles, one feature, several rules and each rule is described by examples. Why is so important to write stories such a way? There are fewer chances to misunderstanding if every statement would be followed by a "formalized" example.
Branches
It may be not obvious, but when you prepare a demo, you'd better have a branch for each section. You would be able to jump over these branches in short time periods and listeners wouldn't fall asleep.
Initial structure
So, first of all, I made an initial structure. It's useful to have something without any business logic, just to check that everything works fine. I use Javalin for hackathons and demos because it blazing fast in development and in startup time. If I would use Spring on this demo, every step would take +20 second just to start context.
First scenario
Then I wrote the first scenario - it was not perfect and this is quite important. If you would try to make a perfect example, you could screw up and freeze on very first steps. My advise - use some "real" examples, but forget about perfection.
^Regular expression$
When you use Cucumber it would suggest you "boilerplate" of step definitions. But JVM implementation would generate it in Java even if you use Kotlin. Before this issue resolution it's a little pain, but better than nothing. So during this step I faced the problem and a fact, that I'm not good at regular expressions. Thanks Lord I found this beatiful cheat sheet which made my work with step definitions simple. There is a way to use cucumber expressions instead of regular expressions, but I think the usage of regular expressions make you so cool ;-)
Bright and shiny Java world
JVM implementation of Cucumber doesn't provide OOB state management between step definitions and I found a good blog post about the usage of Pico Container as DI for state propagation.
Do you think that's all? No, Java, in general, is cursed with too many implementations, look at this list of JSON libraries for Java. khttp which I use for tests of REST API depends on pretty simple and general implementation:
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20150729</version>
</dependency>
Javalin by default use Jackson and later we will see that even these two Json libraries are not enough.
You should say that I could use something instead of khttp, but the problem is that there is no "in time" standards in Java world and there are too many options (IMHO).
So you can check this branch to look into details.
Simple Sexy Working
Implementation was simple and that's why I like Javalin. No magic - just elegant code. I hate myself from 2014 when I decided to use Spring, but at the same time - it was just timely solution of Java EE worlds hell and not my fault that it growths into default choice for 84% of Java projects (84 is my score on exams in informatics, not a magical number).
Outline
I made a lot of work before return to the scenario and make some improvements. This is a good strategy - I made small piece work first then I moved into details. And new examples worked without grand refactoring because refactoring usually takes place because of technical debt (very popular word combination in 2019, more popular only toxic environment). If you follow TDD and write just enough implementations, you wouldn't refactor them a lot, you would extend and compose.
You can see that I only made changes in StepDefintions, and implementation already covered my needs.
class StepDefs(val testInstance: TestInstance) : En {
private val payments: MutableCollection<Payment> = HashSet()
private val url = "http://localhost:7000"
private val url = "http://localhost:"
init {
Given("^system is started on port (\\d+)$") { port: Int ->
testInstance.app = JavalinApp(port, payments).init()
}
Given("^there are some payments$") {
for (i in 1..10) {
Given("^there are (\\d+) payments$") { count: Int ->
for (i in 1..count) {
payments.add(createTestPayment(i))
}
}
When("^request to list all payments sent$") {
testInstance.result = khttp.get(url = this.url + "/api/payments")
testInstance.result = khttp.get(url = this.url + testInstance.app.port() + "/api/payments")
}
Then("^response containse at least (\\d+) payment$") { totalCount: Int ->
Assert.assertTrue(testInstance.result.jsonArray.length()>0)
Then("^response contains (\\d+) elements$") { totalCount: Int ->
Assert.assertEquals(totalCount, testInstance.result.jsonArray.length())
}
Measure it three times, cut it once
When we talk about production, we should think about metrics. And the next step was to add metrics into the demo application.
Javalin made my life easier. Only 4 lines of code and OOB metrics are here.
Containers
Wrap your application into container - trivial task for 2019 and it was easy to use docker and package my application into it.
Continuous Deployment
A little bit harder was to configure CI/CD - I struggled a bit with secrets and ci.yml, but integration between GitLab and Google Cloud Platform helps a lot. I would recommend this blog if you are interested in CD to Kubernetes, but check comments, the original blog is outdated.
Metrics in your repository
More "fun" was to configure the integration of metrics between GCP Prometheus and GitLab environments. There is a lot of documentation about it, but it outdated and not organized well. So it better to check my commit history where I found a solution by the cost of several mistakes. And you definitely should try to use Autodevops in combination with Helm - this is my next target.
Why you need it - I think it's prettier to use inlined metrics in your repo instead of self-hosted Grafana or similar tool.
Holy Fucking Graal
And it was to boring to write even in Kotlin, so I made a decision to use GraalVM and native compilation. You should definitely look at my miserable attempts.
Java loves memory
To work with native Java you should download GraalVM - just try to do that and tell me what you think on Twitter.
Java loves reflection, Graal don't
Problems in case of Javalin were only on serialization/deserialization of Json. Let's look at build script:
RUN cd /tmp && native-image -jar app.jar -H:ReflectionConfigurationFiles=reflection.json -H:+JNI \
-H:Name=practic-payments --static --delay-class-initialization-to-runtime=io.javalin.json.JavalinJson
We configured JavalinJson to be initialized on runtime and described reflection.json like this:
[
{
"name": "[Lorg.eclipse.jetty.servlet.ServletMapping;",
"allDeclaredFields": true,
"allPublicFields": true,
"allDeclaredMethods": true,
"allPublicMethods": true
},
{
"name": "org.slf4j.impl.StaticLoggerBinder",
"allDeclaredFields": true,
"allPublicFields": true,
"allDeclaredMethods": true,
"allPublicMethods": true
},
{
"name": "com.fasterxml.jackson.databind.ObjectMapper",
"allDeclaredFields": true,
"allPublicFields": true,
"allDeclaredMethods": true,
"allPublicMethods": true
},
{
"name": "io.micrometer.core.instrument.Metrics",
"allDeclaredFields": true,
"allPublicFields": true,
"allDeclaredMethods": true,
"allPublicMethods": true
},
{
"name": "PaymentRequestJsonAdapter",
"allDeclaredConstructors" : true,
"allDeclaredFields": true,
"allPublicFields": true,
"allDeclaredMethods": true,
"allPublicMethods": true
},
{
"name": "PaymentRequest",
"allDeclaredConstructors" : true,
"allDeclaredFields": true,
"allPublicFields": true,
"allDeclaredMethods": true,
"allPublicMethods": true
}
]
Before the deserialization case, everything worked fine even without the last two elements. But when I tried to deserialize my PaymentRequest in a native image, it even failed to compile. I think I did something wrong with reflection description or Kotlin have some problems working in such a mix of technologies.
com.oracle.svm.core.code.CEntryPointCallStubs.com_002eoracle_002esvm_002ecore_002eJavaMainWrapper_002erun_0028int_002corg_002egraalvm_002enativeimage_002ec_002etype_002eCCharPointerPointer_0029(generated:0)
Error: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Unsupported method java.lang.Class.getRawTypeAnnotations() is reachable: The declaring class of this element has been substituted, but this element is not present in the substitution class
To diagnose the issue, you can add the option --report-unsupported-elements-at-runtime. The unsupported element is then reported at run time when it is accessed the first time.
Trace:
at parsing java.lang.System$2.getRawClassTypeAnnotations(System.java:1242)
Call path from entry point to java.lang.System$2.getRawClassTypeAnnotations(Class):
at java.lang.System$2.getRawClassTypeAnnotations(System.java:1242)
at sun.reflect.annotation.TypeAnnotationParser.parseAllTypeAnnotations(TypeAnnotationParser.java:315)
at sun.reflect.annotation.TypeAnnotationParser.parseTypeVariableAnnotations(TypeAnnotationParser.java:218)
at sun.reflect.generics.reflectiveObjects.TypeVariableImpl.getAnnotations(TypeVariableImpl.java:233)
at sun.reflect.generics.reflectiveObjects.TypeVariableImpl.getDeclaredAnnotations(TypeVariableImpl.java:237)
at kotlin.reflect.jvm.internal.structure.ReflectJavaAnnotationOwner$DefaultImpls.getAnnotations(ReflectJavaAnnotationOwner.kt:27)
at kotlin.reflect.jvm.internal.structure.ReflectJavaClass.getAnnotations(ReflectJavaClass.kt:27)
at kotlin.reflect.jvm.internal.structure.ReflectJavaClass.getAnnotations(ReflectJavaClass.kt:27)
at kotlin.reflect.jvm.internal.impl.load.java.lazy.LazyJavaAnnotations.iterator(LazyJavaAnnotations.kt:39)
at kotlin.reflect.jvm.internal.impl.types.SimpleType.toString(KotlinType.kt:128)
at java.lang.String.valueOf(String.java:2994)
at java.lang.StringBuilder.append(StringBuilder.java:131)
at com.oracle.svm.core.amd64.AMD64CPUFeatureAccess.verifyHostSupportsArchitecture(AMD64CPUFeatureAccess.java:165)
at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:138)
at com.oracle.svm.core.code.CEntryPointCallStubs.com_002eoracle_002esvm_002ecore_002eJavaMainWrapper_002erun_0028int_002corg_002egraalvm_002enativeimage_002ec_002etype_002eCCharPointerPointer_0029(generated:0)
I decided to use Gson - it was a little bit better, native image compiled but failed at runtime.
Then I found com.squareup.moshi - it's android oriented library to work with Json and it uses code generation and supports Kotlin - greats?
After some angry tweets I made it "work" - it compiled, it ran my application and even process PUT request with deserialization. But if java -jar deserialize it with normal values, the native image just put empty fields to, wait for it, not nullable fields!
Holy --shit-- graal, that was so bad...
Talk, demo and my thoughts
It was my second public demo (hackathons didn't count) and it was good (I guess) - preparation was interesting because of hype technologies, but it was controversial to the talk's point - be software developer, not a coder. Use good tools without hype and it was the last of Java world to convince me and it failed. Goodbye Java my old friend, thank you and farewell.