DevOps #1: Introduction DevOps
Hello I am a full-stack Software Developer and this is my story on how I learn DevOps (development and operations), there will be a bunch of stories I will share with you. Where are we gonna start? Let we dig in this story.
As a full-stack developer, I began to learn a step by step all of the technology that should I learn. And DevOps is one of the important tech practices that I used to know to become a full-stack developer.
DevOps is a culture or workflows environment. A long time ago, software developer and system admin/operations work separately. The software developer does scripting until packaging apps and then sysadmin/ sysops do some build infrastructure and deployment application. There were so many errors when deploying a packaged app, take a long time to do a deployment application process, and usually, they do manually to create a script deployment for a specific application with some manual instructions that so many pages to read. Today, we can find some people or community using and develop a solution for deployment and operation to make an easier and shorter system development life cycle called DevOps. DevOps is kind of culture or workflows environment application development (in my opinion) is to shorten SDLC (System Development Life Cycle) process by implemented automation used CI (Continous Integration) or CD (Continous Delivery). There are some interested isn’t?
And how to become a DevOps engineer? What kind of skill we must learn? This question has 2 points of view. If you’re a Sofware Developer, You must learn some Operations tools that used by SysAdmin/SysOps such as Linux Command Line and etc. And If you are a SysAdmin/ SysOps you must learn some scripting skills to create an automation process.
The Important part of DevOps is Continous Delivery (CD). We can find some continuous delivery tools like Jenkins, Gitlab CI, Circle CI and etc for automation operations. There will be a lot of discussion about it, so I will begin on how developer behaviour on the modern development life cycle. Usually, a programmer will submit their code on some repository like Git, svn, mercurial or another version control system (VCS). And then DevOps engineer will integrate their repository by using a tool. The tools that I mentioned before is a continuous integration tool. Continuous Integration tools will play a role as a scheduler that can do an auto periodically configure and integrate programmer source code repository on the machine to make an update (pull, fetch, etc) newer source code automatically and it automatically triggers build process.
The build process will call build-script that configured or scripted by DevOps engineer by using Gradle/maven for java, composer for PHP, Npm for NodeJS and etc. After that, build process also call an automation test that we can create by our own script to make sure which test is necessary to reach our expectation such as unit test, integration test, functionally test, performance test, stress testing, and security test. And Continous Integration tools are offering us to create some checking script process to check the result is passed or failed. If the test is passed, then our own scripted continuous tools will make a deploy process to our testing server or production server which is we can decide automatically or manually deployed. If it deployed automatically, it’s called Continous Delivery. And If it deployed manually, it can be called Continous Integration.
I will explain what’s that means and different definition between CI/CD. Continuous Integration is an automation process that we create and configure to integrate programmer source code to our machine environment/ VPS (virtual private server), do some test and produce a package deployment. Continuous Delivery is a whole process from integration until the deployment process on testing server/ production server automatically. Now, what is a package deployment?
A package deployment is a package that consists of some component like application (source code, icon, own CSS), dependency(library), configuration (config build to testing or production), and migration script(database migration script to integrate modification database schema), platform (like docker or virtual machine). The four of it is an important component that mandatory should provide (following Continous Delivery standard). The other one is a platform, it’s optionally included but it’s interesting to learn.
What kind of interest thing about platform component that I talking about? It can shorten our effort and it more simple to create a deployment package. We will begin on what is Platform about. Some time ago, the package deployment just consists of application, dependency, configuration and migration script. Nowadays, we also can include a platform or we can say container image. A platform or container image is consist of a single operating system used by our application on our package development. In other words, a container image is a VPS that generated to a file image.
Container image can be replicated, it is an advantage to scaling up our application. Container image can replicate based on our needs and capacity. When there is a lot of request from many users, container image can manage to replicate and created automatically integration process to our machine that recently we provide. It can distribute a heavy burden on our application automatically to another machine we provided to overcome it. Docker has a tool called Docker Swarm to do it, it can manage container replicate and clustered automatically to a new machine. But, there is a popular tool called Kubernetes that provided by Google that many developers believe Kubernetes better than Docker Swarm. But the replication process also has a harm situation to our application. If we aren’t careful we will lose our data and causes an error on our application. But we have a solution to create cluster ready application, such as we must not place a local file to our local service or machine. A local file usually called App Local. What is App local? It’s a file that we store on our local service or machine. App local is divided into two categories, i.e. Memory and File category. Memory category is consist of static variable and session variable. File category is consist of user file upload and log file.
How to Deal With It? How we can not to stored local file to our local machine? Fortunately, there are many services that provide a file service or we can create our own file service. For user storage we can use amazon ES3 file service, google storage or we can use min.io to create our private file service. And where we should store our log file? we can submit it to log aggregator like Splunk or ELK (Elasticsearch Logstash and Kibana) stack. After we have not stored local file we can implement Kubernetes service for clustering our container.
The final stage in our development life cycle is the monitoring process. This process let in a tester, user and operator itself. What kind of monitoring process that we will do? We will monitor resource usage (monitoring CPU, RAM, and I/O), performance (response time, throughput, etc), and application behaviour (log file, distributed tracing for microservice, etc). The good functioning of the equipment, networks and systems will be the key for the business to continue operating. And monitoring systems are responsible for controlling the technology used by the company in order to analyze their operating and performance to detect and alert about possible errors.
That’s all my introduction about DevOps environment and what should we learn to become DevOps engineer :)
