Java 9, 10, 11 - Highlights of new features and enhancements

If you are new to Java, please refer my Newto Java - Getting Started Guide post and previous post about Java8 - Highlights of new features and enhancements?

This could be one of my shortest blog ever! This just contains URLs for sample code snippet (along with easy to understand comments) to learn new features and enhancements of Java 9, Java 10 and Java 11 at my Learning-Java Repository on Github @ 

• Java 9 - https://github.com/tirthalpatel/Learning-Java/tree/master/Java9

• Java 10 - https://github.com/tirthalpatel/Learning-Java/tree/master/Java10

• Java 11 - https://github.com/tirthalpatel/Learning-Java/tree/master/Java11

Microservices with Spring Cloud - Learn by plain-vanilla code snippet

Are you looking for plain-vanilla code with lucid README instructions to learn Spring Cloud technology stack?

Well, I had explored, experimented and experienced building microservices using Spring Cloud almost a year ago. I may upgrade my Spring Cloud learning code to match latest version and include more features of it in this year.

Meanwhile if anyone is interested in below learning goals, the code is available @ https://github.com/tirthalpatel/Learning-Spring/tree/master/gs-spring-cloud/gs-microservices

Spring Boot - Learning made easy by simple application

I hope, below mentioned plain-vanilla application would help you to quickly get started with Spring Boot and few supporting technologies/tools. Additionally, you may experiment Docker, Docker Compose and Pivotal Cloud Foundry, if not yet!

I am not writing step by step instructions again here, because README talks a lot @ https://github.com/tirthalpatel/Learning-Spring/tree/master/gs-spring-boot/gs-spring-boot-app

Learning Objectives

Try following with Spring Boot & run app by deploying on local machine / using docker compose / using docker / to pivotal cloud foundry.

• Spring Boot + Spring Web (MVC and REST) + Thymeleaf (Template engine)
• Spring Boot + Swagger UI (easily visualize and interact with the API’s resources)
• Spring Boot + Properties customization
• Spring Boot + Spring Data JPA + H2 Database + FlywayDB (deal with RDBMS)
• Spring Boot + Spring Data REST + HAL browser (easy to build hypermedia-driven REST web services)
• Spring Boot + Spring Data Redis (publish and subscribe to messages sent via Redis)
• Spring Boot + Production-ready features of Spring Boot Actuator
• Spring Boot + Hawtio (a lightweight web console to monitor and manage application)
• Spring Boot + Developer Tools (for automatic restart, livereload support..)
• Spring Boot + Lombok (a java annotation library to reduce boilerplate code)
• Spring Boot + Testing (JUnit, Hemcrest, AssertJ, Mockito, Spring Test)

How to run the application?

Refer README for instructions to build, deploy and run application using below options. You must need Redis server for option 1, 2 and 3. For option 4, you don't need to set up Redis explicitly, as you can leverage Redis service from Pivotal CF marketplace.

1. Using maven on local machine
2. Using Docker Compose
3. Using Docker
4. Deploying to Pivotal Cloud Foundry (Cloud PaaS)

Java power tools series - Executive diagram

I hope, you have already read "Preface" of my java power tools series.

In this blog, I just thought to publish my one of old mind mapping diagrams (not updated in last 2 years). I think, following java tools classification view should be effective and useful to others' for quick reference.

(Right click the image - Open in New Window - Zoom it)

Please share your experience in comment, if you have something to add / update with respect to following outlined java tools blueprint.

Disclaimer

I am not biased to particular free or commercial tools, rather my objective is about sharing my own experience on set of tools. 

WireMock - A simple standalone HTTP server for service virtualization

I hope, you have already read "Preface" of my java power tools series. In this post, I intend to introduce WireMock tool.

Straight from the horse's mouth

WireMock

• Mock your APIs for fast, robust and comprehensive testing using WireMock (a simulator for HTTP-based APIs /  service virtualization tool / mock server)
• Enables you to stay productive when an API you depend on doesn't exist or isn't complete
• Supports testing of edge cases and failure modes that the real API won't reliably produce
• Features = HTTP response stubbing, request verification, proxy/intercept, record/playback of stubs and fault injection
• Flexible deployment options - run it from within your Java application, JUnit test, Servlet container or as a standalone process
• Powerful Request Matching - Match request URLs, methods, headers cookies and bodies using a wide variety of strategies & first class support for JSON and XML
• Record and Playback - Get up and running quickly by capturing traffic to and from an existing API

My Personal Feedback

I witnessed that WireMock's is extremely easy to setup and harness.  This tool can benefit appreciably in MicroServices Architecture. Let me explain two scenarios, where I could take advantage of WireMock.

(1) Create quick prototypes of REST APIs

Many times, services are not implemented by REST server apps or made available by external vendors. Still, there is a pressing need to develop REST client app beforehand, may be for demo purpose or meeting deliverable goals or due to any other business need. Well, WireMock can help here to create prototypes of agreed APIs quickly, which can be consume by client apps even without realizing these are stubbed APIs instead of real. See running wiremock as a standalone process option.

(2) Record real external services and playback

At times, client app needs to consume third-party or external services over internet or network. However, there are frequent connectivity issues or high latency problem or only limited requests are supported per day due to chargeable services, specifically while accessing these services in Development. Obviously, this may result in high developer productivity loss. Well, WireMock has unique ability to record real services (requests and responses) without a need of single line of coding and playback those in even offline mode automatically. WireMock's this record/playback of stubs feature can greatly boost developer productivity. See record and playback option.

Other scenarios you might want to consider WireMock for:

• Testing mobile apps that depend on third-party REST APIs
• Injecting otherwise hard-to-create errors in 3rd party services
• Any unit testing of code that depends on a web service

All in all, WireMock is a Java-based flexible library with a JSON API that can be used to virtualize, mock, and stub web services.

Try WireMock Practically

Now let's understand fundamental ability of WireMock by running it as a standalone HTTP server and stubbing web services. Let's see practically how effortless it is.

ABC of WireMock standalone server

Download WireMock jar latest version @ http://repo1.maven.org/maven2/com/github/tomakehurst/wiremock-standalone/

To POST data to WireMock's REST endpoint (i.e. http://localhost:9999/__admin/), you can use curl or postman tool or any other choice of tool.

• Start wiremock as a standalone server:  java -jar wiremock-standalone-2.6.0.jar --port 9999 --verbose true
• This would create "__files" and "mappings" folders (in the same folder where wiremock jar exists) to store stubbed services. To see what all service stubs exist on WireMock server (within "__files" and "mappings" folders), open admin console in browser: http://localhost:9999/__admin/
• To stop wiremock, just kill the process or post a request with an empty body to this REST endpoint: http://localhost:9999/__admin/shutdown
• For more detail, see @ http://wiremock.org/docs/running-standalone/.

Stubbing Services / Create REST APIs Prototypes

A core feature of WireMock is the ability to return canned HTTP responses for requests matching criteria. These criteria can be defined in terms of URL, headers and body content.

• Post a stub mapping via the JSON API to this REST endpoint (not persisted, so will lose this after restarting server): http://localhost:9999/__admin/mappings/new
• Stub mappings which have been created can be persisted to the mappings directory via a POST request with an empty body to this REST endpoint: http://localhost:9999/__admin/mappings/save
• Open admin console in browser to validate the added stub mappings: http://localhost:9999/__admin/
• Access mock REST services in browser: http://localhost:9999/hello-world
• To reset all mock request url mapping and response data, just delete all files from "mappings" and "__files" folders.

• For more detail, see @ http://wiremock.org/docs/stubbing/

Record external services and Playback

WireMock has the ability to create stub mappings by recording them while you send requests. This can be used to quickly capture a collection of responses from a real service then use them offline in your tests.

For Example, run this command to record and playback Github Developer APIs:

• Run WireMock server to proxy all requests of Github: java -jar wiremock-standalone-2.6.0.jar --proxy-all="https://api.github.com" --port="9999" --record-mappings
• Access Github APIs via WireMock: http://localhost:9999/users/tirthalpatel

For any requested REST service request (i.e. http://localhost:9999/users/tirthalpatel), first of all WireMock would check, if corresponding stub mapping exists in "mappings" folder or not. If existing, then don't hit real service rather serve recorded data from "mappings" and "__files" folders. If not existing, then WireMock calls external real service (i.e. https://api.github.com/users/tirthalpatel) followed by records stub mapping followed by returns response to the client. That means, we can expect WireMock to playback all recorded stub mappings even if internet is unavailable.

The above explained and few more examples, you can find @ https://github.com/tirthalpatel/Learning-Tools/tree/master/gs-wiremock

Also See

• https://www.infoq.com/articles/Wiremock-testing-mocking-over-wire-stubs

Disclaimer

I am not biased to particular free or commercial tools, rather my objective is about sharing my own experience on set of tools. 

Say "Hello" to Elastic Stack and How to get started with Elasticsearch 5.x?

Last Saturday, I got chance to attend "Log Aggregation and Analysis using Elastic Stack" meetup, organized at Ishi Systems, Ahmedabad. Thanks to Dhaval Mehta, speaker of this meetup, who imparted excellent presentation of this topic along with practical demo.  My key Learning from this meetup is: To accomplish common use cases like Log aggregation and analysis, it's all about investing initial efforts for installation and configuration of open source Elastic Stack (Beats, Logstash, Elasticsearch and Kibana). Apparently, no development efforts are required.

Then, after exploring and playing with Elastic Stack for few hours, I thought to write this post today. In my view, with the release of Elastic Stack 5.0.0 in October 2016, now getting started with Elasticsearch and associated products is a piece of cake.

Everything (Elasticsearch, Kibana, Beats, Logstash, and X-Pack) is well aligned, tested and released together in Elastic Stack. What does it mean to newbie? Well, earlier individuals were responsible to choose, integrate and test best suitable version of products or plugings to accomplish overall Elasticsearch based solution. For example, which plugin to use with Elasticsearch for monitoring, mavel or head plugin? Which version of Logstash or of any other supported tooling will work impeccably with Elasticsearch 1.x or Elasticsearch 2.x? Now, with Elastic Stack version 5.0 onwards, one gets production ready harmonized products. That means, overall experience to deploy and manage Elastic Stack is anticipated to be stable and more graceful than earlier. 

The Open Source Elastic Stack = Reliably and securely take data from any source, in any format, and  search, analyze, and visualize it in real time. Be aware, X-Pack is commercial offering.

Products	Description (see for detail)	Essential Information
Kibana	Visualize your data and navigate the Elastic Stack using Kibana. Kibana gives shape to your data and is the extensible user interface for configuring and managing all aspects of the Elastic Stack.	An open source analytics and visualization platform designed to work with Elasticsearch. Offers easy to use UI for performing advanced data analysis and visualization of data in variety of charts, tables, maps. What if some exceptional use case is not supported out-of-the-box in Kibana? - Well, need to consider Kibana plugin development using "Node, Javascript, HTML and CSS".
Elasticsearch	Store, search, and analyze your data using Elasticsearch. Elasticsearch is a distributed, JSON-based search and analytics engine, designed for horizontal scalability, reliability, and easy management.	Elasticsearch is distributed document store and search server, which offers standard RESTful APIs and JSON to interact with it via client libraries in many languages such as Java, Python, .NET, Groovy, etc. In case of Java application, one may consider Spring Data Elasticsearch for interacting with Elasticsearch. Typical use cases of Elasticsearch are: (1) Elasticsearch as the primary back end (document-oriented NoSQL data store with near real-time search and analytics capabilities) for an application (2) System using another database as the "source of truth", and added Elasticsearch to an existing system to offer enterprise search (3) Using Elasticsearch with existing tools, such as ELK as a large-scale logging framework, now branded as Elastic Stack.
Logstash	Centralize, Transform & Stash Your Data using Logstash. Logstash is an open source, server-side data processing pipeline that ingests data from a multitude of sources simultaneously, transforms it, and then sends it to your favorite “stash.” It has a rich, extensible plugin ecosystem and strong Elasticsearch synergy.	Logstash ecosystem already has 200+ plugins to mix, match, and orchestrate different inputs, filters, and outputs to work in pipeline harmony. (see support matrix) What if ready-to-use-plugin is not available for specific custom need? Well, consider to develop a custom Logstash plugin using "Ruby".
Beats	Lightweight Data Shippers. Beats is the platform for single-purpose data shippers. They install as lightweight agents and send data from hundreds or thousands of machines to Logstash or Elasticsearch.	The Beats family mainly includes (1) Filebeat (for real-time insight into log data) (2) Metricbeat (to ship and analyze metrics) (3) Packetbeat (to analyze network packet data) (4) Winlogbeat (to analyze windows event logs). Additionally, there are numerous community beats are available. What if ready-to-use Beat is not available for specific custom need? Well, consider to develop a custom Beat using "Go".
X-Pack	Security (formerly Shield), Altering (via Watcher), Monitoring (formerly Marvel), Graph and Reporting using X-Pack.  X-Pack has features built and maintained by Elastic engineers that integrate across the Elastic Stack.	Premium offering of Elastic Stack 30 day trial license and then need to purchase a subscription

Also, available as Elastic Cloud service offering.

How to get started with Elasticsearch 5.x?

First of all, understand basic concepts of Elasticsearch (cluster, node, index, type, document, shards and replicas) + see slide @ Elasticsearch as a search alternative to a relational database + watch webcast @ Your Data, Your Search, Elasticsearch.

Try fundamental Elasticsearch REST APIs for exploring cluster, modifying your data and explore your data practically. For this, mainly two options available:

• Install Elasticsearch on your local machine
• Leverage 14 days trial of Elastic Cloud

Option 1: Install Elasticsearch on your local machine

For learning Elasticsearch for long term interest, installing it locally could be good option. Here, you have multiple options.

1. Just install Elasticsearch and try REST APIs using your choice of REST client tool such as postman, curl, etc.
2. Use Elasticsearch docker image, if you are fan of docker.
3. Install Elasticsearch -> Kibana -> X-Pack. Personally, I prefer this option.

If you install Elasticsearch, Kibana and X-Pack, then login Kibana (http://localhost:5601/) using default username/password (elastic / changeme). Then, use "Dev Tools" to play with Elasticsearch REST APIs & see how Elasticsearch clusters, nodes and indices react in "Monitoring" section.

Option 2: Leverage 14 days trial of Elastic Cloud

If you don't wish to install Elasticsearch locally, may be because you are just interested in exploring fundamentals, then just go for leveraging Elastic Cloud to get started with it.

1. Sign up to Elastic Cloud and Login
2. Create new cluster and preserve the password which you get at the end
3. Open Kibana UI and login using the password which you got in previous step, and use "Dev Tools" section to play with Elasticsearch REST APIs

Disclaimer

I am not biased to promote any particular free or commercial products offering, rather my objective is limited to share my own experience.

Docker Toolbox on Windows: Mount any host directory (other than Users directory in c drive) as a data volume

This post is for you, if you are looking for mounting a host directory (which is not underneath the "C:\Users" folder) as a data volume in docker container using Docker Toolbox on Windows.

Problem Statement:

If you are trying to mount a host directory (other than any folder not underneath the "C:\Users"folder) as a data volume specifically using Docker Toolbox on Windows, then it's quite tricky to do.

As per docker documentation --- "If you are using Docker Machine on Mac or Windows, your Docker Engine daemon has only limited access to your OS X or Windows filesystem. Docker Machine tries to auto-share your /Users (OS X) or C:\Users (Windows) directory. So, you can mount files or directories on OS X using".

This means, you can mount anything underneath "c:\Users" directory of host your Windows machine as a data volume in docker container seamlessly. For example, let's say you want to dump some data from "c:\Users\tirthalp\data" directory to "data" directory within Redis container in docker. Well, below just executing command will enable you to do it.

docker run -v /c/Users/tirthalp/.docker/data:/data --name <container-name> <image-name>

But, what is there is a need of mounting other folders (any folder which is not underneath "c:\Users" directory) of your Windows machine as a data volume in docker container? Well, that's not a piece of cake. For example, if you execute below command, the container would not complain. But actually mounting of folder in D drive as a data volume could not happen and data from your host Windows machine would not be accessible in docker container.

docker run -v /d/x-temp/data:/data --name <container-name> <image-name>

Solution:

I could not locate official docker documentation on this. After giving few tries, finally these steps worked --- add a shared folder to Docker Host VM and mount it as a data volume. Although the given steps are pretty clear, I thought to share screenshots of the same as following.

Step 1: Stop Docker Machine

Step 2: Add Shared Folder in Oracle VirtualBox Manager

Open Oracle VM VirtualBox Manager. Settings -> Shared Folder -> Add Share Folder (i.e. Tirthal-LABs).

Step 3: Permanently mount VirtualBox shared folder as a shared-data volume in Docker machine

Start docker machine.

SSH to docker machine, create directory in docker machine (i.e. /home/docker/tirthal-projects) and mount VirtualBox shared folder name (i.e. Tirthal-LABs) with it. This should enable you to access host Windows machine's data in docker machine @ /home/docker/tirthal-projects.

Try restarting docker machine. Now if you check at /home/docker/tirthal-projects, you would not see mounted host machine's data. For permanent mounting, add following two lines at the end of profile --- sudo vi /mnt/sda1/var/lib/boot2docker/profile

mkdir /home/docker/<directory-name>

sudo mount -t vboxsf -o uid=1000,gid=50 <virtual-box-shared-folder-name> /home/docker/<directory-name>

Try restarting docker machine again. Now if you check at /home/docker/tirthal-projects, you would not see mounted host machine's data.

Step 4: Mount a shared-storage volume as a data volume

Now you should be able to mount even other than "c:\Users" directories of host Windows machine in docker container as a data volume, if you have completed the above steps 1 to 3 successfully. For example, following command would mount "data" folder in container with a shared-storage "/home/docker/tirthal-projects/temp" on docker machine. Here, the "/home/docker/tirthal-projects" is mounted with the host Window machine's folder (i.e. D:\Tirthal-LABs\).

docker run -v /home/docker/tirthal-projects/temp:/data --name <container-name> <image-name>

References:

• https://docs.docker.com/engine/installation/windows/
• https://docs.docker.com/engine/tutorials/dockervolumes/#/mount-a-host-directory-as-a-data-volume
• http://www.erikojebo.se/Code/Details/791

Liferay Portal 7 - Getting Started Guide Series

I had written step by step post for LiferayPortal 6.2 - Getting Started Guide - Development Setup using Eclipse Kepler in past. Today, I thought to write series of posts for getting started with recently released Liferay Portal 7 CE, which has major changes with respect to introducing modular architecture and related development strategy.

If you are new to portal, then read what is portal? If you are new to Liferay, then read what is Liferay Portal (now broadly known as Liferay Digital Experience Platform)? But most of time, we don't understand all theoretical fundamentals until we have environment to try something practically. So first attempt can be to setup latest Liferay portal to try its few out-of-box features and functionalities. Then, further attempts can be setup Liferay IDE for creating hello world project and deploying to Liferay Portal.

Following series of posts may help you to get started with Liferay Portal 7 quickly, if you already have experience with Java development using Eclipse.

1. Step by step guide to setup Liferay Portal
2. Understanding Liferay Development Tooling options
3. Basic setup of Liferay IDE
4. Using Liferay IDE for Hello World Liferay MVC Portlet development and deployment

For now, enjoy these posts. I may add few more to the above list in future!

Also Refer

• Liferay Portal 7.0 CE Release Announcement
• What's changed and what hasn't in Liferay 7

• Liferay 7 Modular Source Code

• Liferay 7 Features from Developer Perspective
• OSGi Support
• Bootstrap 3.0 Look and Feel
• Java 8 Support
• Alloy Editor
• Elastic Search
• Support Integration Testing through Arquillian
• Application Display Templates Support for Login, Language and Breadcrumb portlets.
• Improvements in Web content management
• Singe Page Application Development
• Document Management storages extracted as modules      
• Service Builder code now uses Declarative Services instead of Spring for dependency injection

Liferay Portal 7 - Getting Started Guide - Hello World Portlet Module using Liferay IDE

This post is part of Liferay Portal 7 Getting Started Guide Series.

Now, it's time to get started with Liferay IDE for hello world liferay mvn portlet module development and deployment to Liferay Portal by following below quick steps.

Liferay Workspace and Module Project = Newly introduced concept of Liferay 7, and were not in Liferay 6.2.x or prior versions.

Prerequisite

• Basic Setup of Liferay IDE

Create Workspace Project

Liferay Workspace can be used to hold and manage Liferay projects, which is the official way to create Liferay 7.0 modules using Gradle. Let's create it using Liferay IDE -> New Liferay Workspace Project -> Give Workspace name and Finish. This should switch you to "Liferay Workspace" perspective, in case it was not by default. (In case you lose Liferay Perspective, then set it again using Window -> Perspective -> Other Perspective..)

While creating new workspace, don't select "Download liferay bundle" option, as we consider to deploy our hello world module into locally running Liferay portal instance which is outside of workspace. Alternatively, if you wish to auto generate liferay within workspace and deploy modules there, then this option can be used.

Note: If you’ve already created a Liferay Workspace and you’d like to import it into your existing IDE, you can do so by navigating to File -> Import -> Liferay -> Liferay Workspace Project. Then, click Next and browse for your worspace project. Once you’ve selected you workspace, click Finish.

After creating workspace project, let's proceed to create Hello World module project.

Create Hello World MVC Portlet Module Project

Select "modules" folder in workspace and use New Liferay Module Project option to create Liferay Module. Give your choice of project name (i.e. hello-world-mvc-portlet), select "mvcportlet" in Project Template name and click Finish. That's it. At this point we have our structure and our portlet default code underneath the modules folder in the workspace. Just quickly explore auto-generated code.

Note: If you look at the @ Component's property and you’ve been a Liferay developer, these should look familiar as they were once XML attributes that were previously in liferay-portlet.xml, liferay-display.xml, and portlet.xml. The options available are in liferay-portlet-app_7_0_0.dtd.

Refer Liferay documentation for more detail. Now, let's proceed to deploy it to Liferay Portal.

Deploy to Liferay Portal

First of all, start the Liferay portal server, if not yet!

IDE's "Liferay Workspace" perspective has "Gradle Tasks" view (Gradle is build tool, in case you don't know). Just click "deploy" option underneath your module's build category. This should generate "<your-module>.<version>.jar" file in "build/lib" folder underneath your module project folder within workspace. Ideally, this step should automatically deploy this jar bundle in locally running liferay portal on your machine (the way it happens in "blade deploy" command), however, somehow it's not deploying to liferay portal on my local setup! Therefore, I'll be showing how to deploy it manually in next step.

For manual deployment, just copy the generated portlet's jar file into "deploy" folder underneath the Liferay Portal Home folder. Then, this jar would disappeared automatically from deploy folder within few seconds. In console, you should see this message "… STARTED <your-portlet-module> [bundle-id]". Actually, Liferay installs this jar underneath "<TOMCAT_HOME>/osgi/modules" folder of Liferay Portal. See documentation for more detail.

Add custom Portlet on Site Page of Portal

Login to Liferay Portal as admin user. Add your hello world application on page (Add -> Applications -> Sample -> add portlet application). Additionally, try few text code changes in "view.jsp" file of hello world module and deploy again to see how it goes. At the end, if you wish, you can easily "Remove" custom portlet app from the page too.

Managing Liferay Modules Projects Lifecycle

Liferay 7.0 architecture offers modularity via OSGI. That means, different modules, so called OSGI bundles, can have their own lifecycle to start, restart, stop, etc. That can be managed using any of below options.

• "Gogo Shell" in IDE. Right-click the started portal in server view and select Open Gogo Shell. For example, one can see list of bundles containing "hello" word using this command: lb -s | grep "hello". See Gogo Shell commands documentation.
• Liferay Administration. Control Panel -> Apps -> App Manager. See documentation for more detail.

What's next?

As an alternate tooling of Liferay IDE, you may try Blade CLI tool for hello world liferay mvc portlet module development and deployment to Liferay Portal!

1. Install Blade CLI.
2. Create Workspace: blade init <workspace-name>
3. Create Hello World MVC Portlet Application as a Liferay Module: blade create -t mvcportlet -p [package name] -c [class name] [project name]
4. At this stage, you would have Gradle based project structure and initial code. Do necessary code changes using your choice of IDE.
5. Ensure local liferay portal instance is running on your machine. Deploying Module to Liferay Portal: blade deploy
6. Add application on site page by logging to liferay portal as admin user