<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" xmlns:media="http://search.yahoo.com/mrss/"><generator uri="https://jekyllrb.com/" version="4.4.1">Jekyll</generator><link href="https://mehmandarov.com/tag/automation/feed.xml" rel="self" type="application/atom+xml"/><link href="https://mehmandarov.com/tag/automation/" rel="alternate" type="text/html"/><updated>2019-03-01T07:01:00+01:00</updated><id>https://mehmandarov.com/tag/automation/feed.xml</id><title type="html">Rustam Mehmandarov - tag: automation</title><subtitle type="text">Posts tagged &quot;automation&quot; on Rustam Mehmandarov.</subtitle><author><name>Rustam Mehmandarov</name></author><entry><title type="html">Configuring Slack Notifications for Google Cloud Build</title><link href="https://mehmandarov.com/slack-notifications-for-cloud-build/" rel="alternate" type="text/html" title="Configuring Slack Notifications for Google Cloud Build"/><published>2019-03-01T07:01:00+01:00</published><updated>2019-03-01T07:01:00+01:00</updated><id>https://mehmandarov.com/slack-notifications-for-cloud-build</id><content type="html" xml:base="https://mehmandarov.com/slack-notifications-for-cloud-build/"><![CDATA[<p><em>Adding Slack notifications for your Google Cloud Build jobs explained with code examples and screenshots.</em></p>

<ul>
  <li><a href="#intro">Intro</a></li>
  <li><a href="#1-before-you-begin">1. Before you begin</a></li>
  <li><a href="#2-create-a-cloud-function">2. Create a Cloud Function</a></li>
  <li><a href="#3-deploy-the-cloud-function">3. Deploy the Cloud Function</a></li>
</ul>

<hr />

<h2 id="intro">Intro</h2>

<p>Recently we decided to migrate our builds from Travis CI to Google Cloud Build to speed up the builds. The process was quite easy and flawless; however, we were still missing a few minor things. One of them was the notifications from Cloud Build to our <code class="language-plaintext highlighter-rouge">#ops</code> channel in Slack. This was slightly annoying because you would not know if the build was finished and the site was deployed, or if it failed for some reason.</p>

<p>Integrating with Cloud Build was a bit more different than what you are used to from integrations with Jenkins or Travis CI. Normally you would just create a webhook that would call an interface in the Slack API. In Cloud Build, on the other hand, everything is getting posted to the Pub/Sub queue built into the platform, and here you would just need to subscribe to the specific queue and listen for the events. To achieve the latter, you would need a small serverless function to listen for these events and to call the Slack API.</p>

<p><img src="/assets/images/posts-images/2019-03-01-architecture.png" alt="The Architecture" class="bigger-image" /></p>
<figcaption class="caption">The Architecture.</figcaption>
<p><br /></p>

<p>Note that here we will, technically, be using paid services on Google Cloud Platform, as both Cloud Build, Cloud Pub/Sub, and Cloud Functions are billable components. However, since all the components above provide a generous free tier, you will need to work hard to get passed the free tier with this setup.</p>

<ul>
  <li><strong>Cloud Build:</strong> Free first 120 builds-minutes per day for <code class="language-plaintext highlighter-rouge">Basic</code> machine type (<code class="language-plaintext highlighter-rouge">n1-standard-1</code>).</li>
  <li><strong>Cloud Pub/Sub:</strong> Free first 10GB per month (<a href="https://cloud.google.com/pubsub/pricing">pricing</a>).</li>
  <li><strong>Cloud Functions:</strong> Free first 2 million invocations per month (<a href="https://cloud.google.com/functions/pricing">pricing</a>).</li>
</ul>

<h2 id="1-before-you-begin">1. Before you begin</h2>

<h3 id="11-prepare-your-gcp-project">1.1 Prepare your GCP project</h3>

<p><em>I assume you have a Google Cloud Account, and that you have signed in to your account.</em></p>

<ol>
  <li>Select or create a Google Cloud Platform project, e.g. from the <a href="https://console.cloud.google.com/cloud-resource-manager">Manage resources page</a>.</li>
  <li>Make sure that <a href="https://cloud.google.com/billing/docs/how-to/modify-project">billing is enabled</a> for your Google Cloud Platform project.</li>
  <li>Enable the Cloud Functions and Cloud Pub/Sub. You can also enable the APIs using this <a href="https://console.cloud.google.com/flows/enableapi?apiid=cloudfunctions,pubsub">link</a>.</li>
  <li>Use <a href="https://cloud.google.com/shell/docs/quickstart">Cloud Shell</a> right from the browser, or you can <a href="https://cloud.google.com/sdk/docs/">Install and initialize the Cloud SDK</a> on your own machine.</li>
  <li>If you have installed the Cloud SDK, update and install gcloud components:</li>
</ol>

<figure class="highlight"><pre><code class="language-bash" data-lang="bash">gcloud components update <span class="o">&amp;&amp;</span>
gcloud components <span class="nb">install </span>alpha beta</code></pre></figure>

<h3 id="12-prepare-your-slack-app">1.2 Prepare your Slack App</h3>

<p><em>I assume you have Slack installed and that you have created and signed-in to your account.</em></p>

<p>Create a <a href="https://api.slack.com/apps?new_app=1">new Slack app</a>:</p>

<ol>
  <li>Choose the app&#8217;s name and your Slack team. Click Create.</li>
  <li>Click Incoming Webhooks.</li>
  <li>Activate incoming webhooks.</li>
  <li>Click Add New Webhook to Workspace. An authorization page opens.</li>
  <li>From the drop-down menu, select the channel to which you would like notifications sent.</li>
  <li>Click Authorize.</li>
  <li>A webhook for your Slack application has been created. Copy the webhook URL and save it for later use.</li>
</ol>

<h2 id="2-create-a-cloud-function">2. Create a Cloud Function</h2>

<p>We need to create a Cloud Storage bucket to stage your Cloud Functions files. Use <code class="language-plaintext highlighter-rouge">[STAGING_BUCKET_NAME]</code> that is a globally-unique bucket name (such as <code class="language-plaintext highlighter-rouge">[PROJECT-ID]_cloudbuilds</code>):</p>

<figure class="highlight"><pre><code class="language-bash" data-lang="bash">gsutil mb gs://[STAGING_BUCKET_NAME]</code></pre></figure>

<p>You should see the following output:</p>

<figure class="highlight"><pre><code class="language-bash" data-lang="bash">Creating gs://[PROJECT-ID]_cloudbuilds/[STAGING_BUCKET_NAME]...</code></pre></figure>

<p>Next, create a directory on your local system for the application code:</p>

<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">mkdir</span> ~/gcb_slack
<span class="nb">cd</span> ~/gcb_slack</code></pre></figure>

<p>Then, create the following two files in the <code class="language-plaintext highlighter-rouge">gcb_slack</code> directory.</p>

<p><strong>File 1:</strong> <code class="language-plaintext highlighter-rouge">package.json</code></p>

<figure class="highlight"><pre><code class="language-json" data-lang="json"><span class="p">{</span><span class="w">
  </span><span class="nl">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"google-container-slack"</span><span class="p">,</span><span class="w">
  </span><span class="nl">"version"</span><span class="p">:</span><span class="w"> </span><span class="s2">"0.0.1"</span><span class="p">,</span><span class="w">
  </span><span class="nl">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Slack integration for Google Cloud Build, using Google Cloud Functions"</span><span class="p">,</span><span class="w">
  </span><span class="nl">"main"</span><span class="p">:</span><span class="w"> </span><span class="s2">"index.js"</span><span class="p">,</span><span class="w">
  </span><span class="nl">"dependencies"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
    </span><span class="nl">"@slack/client"</span><span class="p">:</span><span class="w"> </span><span class="s2">"4.10.0"</span><span class="w">
  </span><span class="p">}</span><span class="w">
</span><span class="p">}</span></code></pre></figure>

<p><strong>File 2:</strong> <code class="language-plaintext highlighter-rouge">index.js</code></p>

<p><em>Note: Make sure to update <code class="language-plaintext highlighter-rouge">SLACK_WEBHOOK_URL</code> in the code below.</em></p>

<figure class="highlight"><pre><code class="language-javascript" data-lang="javascript"><span class="kd">const</span> <span class="nx">IncomingWebhook</span> <span class="o">=</span> <span class="nf">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">@slack/client</span><span class="dl">'</span><span class="p">).</span><span class="nx">IncomingWebhook</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">SLACK_WEBHOOK_URL</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">&lt;INSERT YOUR WEBHOOK FROM STEP 1.2&gt;</span><span class="dl">"</span>

<span class="kd">const</span> <span class="nx">webhook</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">IncomingWebhook</span><span class="p">(</span><span class="nx">SLACK_WEBHOOK_URL</span><span class="p">);</span>

<span class="c1">// subscribe is the main function called by Cloud Functions.</span>
<span class="nx">module</span><span class="p">.</span><span class="nx">exports</span><span class="p">.</span><span class="nx">subscribe</span> <span class="o">=</span> <span class="p">(</span><span class="nx">event</span><span class="p">,</span> <span class="nx">callback</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
 <span class="kd">const</span> <span class="nx">build</span> <span class="o">=</span> <span class="nf">eventToBuild</span><span class="p">(</span><span class="nx">event</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">data</span><span class="p">);</span>

  <span class="c1">// Skip if the current status is not in the status list.</span>
  <span class="c1">// Add additional statues to list if you'd like:</span>
  <span class="c1">// QUEUED, WORKING, SUCCESS, FAILURE,</span>
  <span class="c1">// INTERNAL_ERROR, TIMEOUT, CANCELLED</span>
  <span class="kd">const</span> <span class="nx">status</span> <span class="o">=</span> <span class="p">[</span><span class="dl">'</span><span class="s1">SUCCESS</span><span class="dl">'</span><span class="p">,</span> <span class="dl">'</span><span class="s1">FAILURE</span><span class="dl">'</span><span class="p">,</span> <span class="dl">'</span><span class="s1">INTERNAL_ERROR</span><span class="dl">'</span><span class="p">,</span> <span class="dl">'</span><span class="s1">TIMEOUT</span><span class="dl">'</span><span class="p">];</span>
  <span class="k">if </span><span class="p">(</span><span class="nx">status</span><span class="p">.</span><span class="nf">indexOf</span><span class="p">(</span><span class="nx">build</span><span class="p">.</span><span class="nx">status</span><span class="p">)</span> <span class="o">===</span> <span class="o">-</span><span class="mi">1</span><span class="p">)</span> <span class="p">{</span>
    <span class="k">return</span> <span class="nf">callback</span><span class="p">();</span>
  <span class="p">}</span>

  <span class="c1">// Send message to Slack.</span>
  <span class="kd">const</span> <span class="nx">message</span> <span class="o">=</span> <span class="nf">createSlackMessage</span><span class="p">(</span><span class="nx">build</span><span class="p">);</span>
  <span class="nx">webhook</span><span class="p">.</span><span class="nf">send</span><span class="p">(</span><span class="nx">message</span><span class="p">,</span> <span class="nx">callback</span><span class="p">);</span>
<span class="p">};</span>

<span class="c1">// eventToBuild transforms pubsub event message to a build object.</span>
<span class="kd">const</span> <span class="nx">eventToBuild</span> <span class="o">=</span> <span class="p">(</span><span class="nx">data</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
  <span class="k">return</span> <span class="nx">JSON</span><span class="p">.</span><span class="nf">parse</span><span class="p">(</span><span class="k">new</span> <span class="nc">Buffer</span><span class="p">(</span><span class="nx">data</span><span class="p">,</span> <span class="dl">'</span><span class="s1">base64</span><span class="dl">'</span><span class="p">).</span><span class="nf">toString</span><span class="p">());</span>
<span class="p">}</span>

<span class="c1">// createSlackMessage create a message from a build object.</span>
<span class="kd">const</span> <span class="nx">createSlackMessage</span> <span class="o">=</span> <span class="p">(</span><span class="nx">build</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
  <span class="kd">let</span> <span class="nx">message</span> <span class="o">=</span> <span class="p">{</span>
   <span class="na">text</span><span class="p">:</span> <span class="s2">`Build </span><span class="se">\`</span><span class="p">${</span><span class="nx">build</span><span class="p">.</span><span class="nx">id</span><span class="p">}</span><span class="se">\`</span><span class="s2">`</span><span class="p">,</span>
    <span class="na">mrkdwn</span><span class="p">:</span> <span class="kc">true</span><span class="p">,</span>
    <span class="na">attachments</span><span class="p">:</span> <span class="p">[</span>
      <span class="p">{</span>
        <span class="na">title</span><span class="p">:</span> <span class="dl">'</span><span class="s1">Build logs - Your Custom Message Goes Here</span><span class="dl">'</span><span class="p">,</span>
        <span class="na">title_link</span><span class="p">:</span> <span class="nx">build</span><span class="p">.</span><span class="nx">logUrl</span><span class="p">,</span>
        <span class="na">fields</span><span class="p">:</span> <span class="p">[{</span>
          <span class="na">title</span><span class="p">:</span> <span class="dl">'</span><span class="s1">Status</span><span class="dl">'</span><span class="p">,</span>
          <span class="na">value</span><span class="p">:</span> <span class="nx">build</span><span class="p">.</span><span class="nx">status</span>
        <span class="p">}]</span>
      <span class="p">}</span>
    <span class="p">]</span>
  <span class="p">};</span>
  <span class="k">return</span> <span class="nx">message</span>
<span class="p">}</span></code></pre></figure>

<h2 id="3-deploy-the-cloud-function">3. Deploy the Cloud Function</h2>

<p>To deploy the subscribe function with a Cloud Pub/Sub trigger, run the following command in the <code class="language-plaintext highlighter-rouge">gcb_slack</code> directory:</p>

<figure class="highlight"><pre><code class="language-bash" data-lang="bash">gcloud functions deploy subscribe <span class="nt">--stage-bucket</span> <span class="o">[</span>STAGING_BUCKET_NAME] <span class="se">\</span>
    <span class="nt">--trigger-topic</span> cloud-builds</code></pre></figure>

<p>where <code class="language-plaintext highlighter-rouge">[STAGING_BUCKET_NAME]</code> is the name of your staging Cloud Storage Bucket that you defined earlier.</p>

<p>You should see an output confirming the creation of the cloud function and <code class="language-plaintext highlighter-rouge">status: READY</code>.</p>

<p>After you&#8217;ve completed deployment of the Cloud Function, when a build event occurs, you will receive a Slack notification.</p>

<p><img src="/assets/images/posts-images/2019-03-01-slack-app.png" alt="The Slack App in action" /></p>
<figcaption class="caption">The Slack App in action.</figcaption>
<p><br /></p>

<p>Also, feel free to customize your app, like adding a custom icon, as I did with mine. &#9757;&#65039;</p>

<hr />]]></content><author><name>Rustam Mehmandarov</name></author><summary type="html">Setting up Slack notifications for Google Cloud Build jobs with code examples and screenshots.</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://mehmandarov.com/assets/images/posts-images/lego-bots.jpg"/><category term="blog"/><category term="field notes"/><category term="automation"/><category term="cloud"/><category term="english"/></entry><entry><title type="html">Automating Confluence Posts with Amazon Cloud (AWS)</title><link href="https://mehmandarov.com/confluence-and-aws/" rel="alternate" type="text/html" title="Automating Confluence Posts with Amazon Cloud (AWS)"/><published>2017-01-29T09:26:00+01:00</published><updated>2017-01-29T09:26:00+01:00</updated><id>https://mehmandarov.com/confluence-and-aws</id><content type="html" xml:base="https://mehmandarov.com/confluence-and-aws/"><![CDATA[<p><em>How do you automate a job without setting up a dedicated machine?</em></p>

<ul>
  <li><a href="#posting-via-confluence-rest-api">Posting via Confluence REST API</a></li>
  <li><a href="#creating-a-lambda-function-in-aws">Creating a Lambda Function in AWS</a></li>
  <li><a href="#schedule-it">Schedule it!</a></li>
</ul>

<hr />

<p>Some mundane tasks can easily be automated. In my case, I needed something that could post to our Confluence instance at a specific time every week. This is because we use blog posts in Confluence for announcing our weekly volunteer meetings and as a simple attendance management.</p>

<hr />

<h2 id="posting-via-confluence-rest-api">Posting via Confluence REST API</h2>

<p><a href="https://en.wikipedia.org/wiki/Confluence_(software)" target="_blank">Confluence</a>, a team collaboration software written in Java and quite often used in corporate environments, offers a <a href="https://developer.atlassian.com/confdev/confluence-server-rest-api/confluence-rest-api-examples" target="_blank">REST API</a> that makes it possible to perform many operations on the content. In this case, I was interested in posting new content with some text, and I was getting tired of doing that manually every week. So, I thought: <a href="https://xkcd.com/1319/" target="_blank">&#8220;I spend a lot of time on this task. I should write a program automating it!&#8221;</a>.</p>

<p>The first take on that problem was a simple Python script:</p>

<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="kn">import</span> <span class="n">json</span>
<span class="kn">import</span> <span class="n">requests</span>

<span class="n">url</span> <span class="o">=</span> <span class="sh">"</span><span class="s">https://some-confluence-url.com/rest/api/content/</span><span class="sh">"</span>
<span class="n">headers</span> <span class="o">=</span> <span class="p">{</span> <span class="sh">"</span><span class="s">content-type</span><span class="sh">"</span><span class="p">:</span> <span class="sh">"</span><span class="s">application/json</span><span class="sh">"</span> <span class="p">}</span>
<span class="n">payload</span> <span class="o">=</span> <span class="p">{</span> <span class="sh">"</span><span class="s">type</span><span class="sh">"</span><span class="p">:</span> <span class="sh">"</span><span class="s">blogpost</span><span class="sh">"</span><span class="p">,</span>
            <span class="sh">"</span><span class="s">title</span><span class="sh">"</span><span class="p">:</span> <span class="sh">"</span><span class="s">Fancy meeting title</span><span class="sh">"</span><span class="p">,</span>
            <span class="sh">"</span><span class="s">space</span><span class="sh">"</span><span class="p">:</span> <span class="p">{</span>
                <span class="sh">"</span><span class="s">key</span><span class="sh">"</span><span class="p">:</span> <span class="sh">"</span><span class="s">mySpaceNameGoesHere</span><span class="sh">"</span>
            <span class="p">},</span>
            <span class="sh">"</span><span class="s">body</span><span class="sh">"</span><span class="p">:</span> <span class="p">{</span>
                <span class="sh">"</span><span class="s">storage</span><span class="sh">"</span><span class="p">:</span> <span class="p">{</span>
                    <span class="sh">"</span><span class="s">representation</span><span class="sh">"</span><span class="p">:</span> <span class="sh">"</span><span class="s">storage</span><span class="sh">"</span><span class="p">,</span>
                    <span class="sh">"</span><span class="s">value</span><span class="sh">"</span><span class="p">:</span> <span class="sh">"</span><span class="s">Meeting description goes here.</span><span class="sh">"</span>
                <span class="p">}</span>
            <span class="p">}</span>
            <span class="c1">#"history": {
</span>            <span class="c1">#    "createdDate": '2017-01-10T14:00:00.000Z'
</span>            <span class="c1">#},
</span>
        <span class="p">}</span>

<span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="p">.</span><span class="nf">post</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> 
                         <span class="n">data</span><span class="o">=</span><span class="n">json</span><span class="p">.</span><span class="nf">dumps</span><span class="p">(</span><span class="n">payload</span><span class="p">),</span> 
                         <span class="n">headers</span><span class="o">=</span><span class="n">headers</span><span class="p">,</span> 
                         <span class="n">auth</span><span class="o">=</span><span class="p">(</span><span class="sh">"</span><span class="s">myUserName</span><span class="sh">"</span><span class="p">,</span> <span class="sh">"</span><span class="s">mySecretPassword</span><span class="sh">"</span><span class="p">))</span>
<span class="k">print</span> <span class="n">json</span><span class="p">.</span><span class="nf">dumps</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="nf">json</span><span class="p">(),</span> <span class="n">indent</span><span class="o">=</span><span class="mi">4</span><span class="p">,</span> <span class="n">sort_keys</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span></code></pre></figure>

<p>You should be able to run this script against your end-point and post your content. Just note that I used <code class="language-plaintext highlighter-rouge">requests</code> library here that needs to be installed separately, for instance, by using <code class="language-plaintext highlighter-rouge">pip</code>:</p>

<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">sudo </span>pip <span class="nb">install </span>requests</code></pre></figure>

<p>Also, you will need to update the following values in the script:</p>

<ul>
  <li><code class="language-plaintext highlighter-rouge">url</code> &#8211; the URL to your site</li>
  <li><code class="language-plaintext highlighter-rouge">type</code> &#8211; can be a <code class="language-plaintext highlighter-rouge">blogpost</code> or a <code class="language-plaintext highlighter-rouge">page</code></li>
  <li><code class="language-plaintext highlighter-rouge">title</code> &#8211;&#160;page or post title</li>
  <li><code class="language-plaintext highlighter-rouge">space</code> &#8211; a Confluence space where the post will belong</li>
  <li><code class="language-plaintext highlighter-rouge">value</code> &#8211; content of the page, can contain HTML tags</li>
  <li><code class="language-plaintext highlighter-rouge">auth</code> &#8211;&#160;username and password (please note that, for security reasons, it might be a good idea to create a separate user with restricted access rights just for posting specific content)</li>
  <li><code class="language-plaintext highlighter-rouge">createdDate</code> &#8211; you can also add the commented-out part to add a specific blog post date</li>
</ul>

<p>The script uses basic authentication to post the content and, if successful, returns metadata about the post as a response. If it fails, the response from Confluence end-point should point you in the right direction.</p>

<hr />

<h2 id="creating-a-lambda-function-in-aws">Creating a Lambda Function in AWS</h2>

<p>Now we are one step closer to automation. The script works, and we can post content with a simple push of a button. What now?</p>

<p>Well, now I needed something to run that script and something to trigger that action. The trigger is the time as I wanted to run our script weekly, but first I needed to deploy our script to the Cloud. Scheduling and triggers would come later.</p>

<p>Since we had some infrastructure on AWS already, I decided to deploy the script on the same platform. I wanted to deploy it as a simple stand-alone function in the Cloud &#8211; in AWS world it is called a <a href="http://docs.aws.amazon.com/lambda/latest/dg/welcome.html" target="_blank">Lambda function</a>.</p>

<p>I started by wrapping our code into a function and making it AWS Lambda function compatible &#8211; to define a function that takes two arguments: <code class="language-plaintext highlighter-rouge">event</code> and <code class="language-plaintext highlighter-rouge">context</code>. I choose to call the function <code class="language-plaintext highlighter-rouge">blogpost_handler</code>:</p>

<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c1">#!/usr/bin/python
# -*- coding: utf-8 -*-
</span>
<span class="c1"># File name: poster.py
</span>
<span class="kn">import</span> <span class="n">requests</span>
<span class="kn">import</span> <span class="n">json</span>
<span class="kn">import</span> <span class="n">datetime</span>

<span class="k">def</span> <span class="nf">blogpost_handler</span><span class="p">(</span><span class="n">event</span><span class="p">,</span> <span class="n">context</span><span class="p">):</span> 

    <span class="c1"># Script is run on Sunday, need to use date for Tuesday
</span>    <span class="n">next_tue</span> <span class="o">=</span> <span class="p">(</span><span class="n">datetime</span><span class="p">.</span><span class="n">datetime</span><span class="p">.</span><span class="nf">now</span><span class="p">()</span>
                    <span class="o">+</span><span class="n">datetime</span><span class="p">.</span><span class="nf">timedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">2</span><span class="p">)).</span><span class="nf">strftime</span><span class="p">(</span><span class="sh">"</span><span class="s">%Y-%m-%d</span><span class="sh">"</span><span class="p">)</span>

    <span class="n">url</span> <span class="o">=</span> <span class="sh">"</span><span class="s">https://some-confluence-url.com/rest/api/content/</span><span class="sh">"</span>
    <span class="n">headers</span> <span class="o">=</span> <span class="p">{</span> <span class="sh">"</span><span class="s">content-type</span><span class="sh">"</span><span class="p">:</span> <span class="sh">"</span><span class="s">application/json</span><span class="sh">"</span> <span class="p">}</span>
    <span class="n">payload</span> <span class="o">=</span> <span class="p">{</span> <span class="sh">"</span><span class="s">type</span><span class="sh">"</span><span class="p">:</span> <span class="sh">"</span><span class="s">blogpost</span><span class="sh">"</span><span class="p">,</span>
                <span class="sh">"</span><span class="s">title</span><span class="sh">"</span><span class="p">:</span> <span class="sh">"</span><span class="s">Meeting: </span><span class="sh">"</span><span class="o">+</span><span class="n">next_tue</span><span class="p">,</span>
                <span class="sh">"</span><span class="s">space</span><span class="sh">"</span><span class="p">:</span> <span class="p">{</span>
                    <span class="sh">"</span><span class="s">key</span><span class="sh">"</span><span class="p">:</span> <span class="sh">"</span><span class="s">mySpaceNameGoesHere</span><span class="sh">"</span>
                <span class="p">},</span>
                <span class="sh">"</span><span class="s">body</span><span class="sh">"</span><span class="p">:</span> <span class="p">{</span>
                    <span class="sh">"</span><span class="s">storage</span><span class="sh">"</span><span class="p">:</span> <span class="p">{</span>
                        <span class="sh">"</span><span class="s">representation</span><span class="sh">"</span><span class="p">:</span> <span class="sh">"</span><span class="s">storage</span><span class="sh">"</span><span class="p">,</span>
                        <span class="sh">"</span><span class="s">value</span><span class="sh">"</span><span class="p">:</span> <span class="sh">"</span><span class="s">Meeting description goes here.</span><span class="sh">"</span>
                    <span class="p">}</span>
                <span class="p">}</span>
            <span class="p">}</span>

    
    <span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="p">.</span><span class="nf">post</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> 
                             <span class="n">data</span><span class="o">=</span><span class="n">json</span><span class="p">.</span><span class="nf">dumps</span><span class="p">(</span><span class="n">payload</span><span class="p">),</span> 
                             <span class="n">headers</span><span class="o">=</span><span class="n">headers</span><span class="p">,</span> 
                             <span class="n">auth</span><span class="o">=</span><span class="p">(</span><span class="sh">"</span><span class="s">myUserName</span><span class="sh">"</span><span class="p">,</span> <span class="sh">"</span><span class="s">mySecretPassword</span><span class="sh">"</span><span class="p">))</span>
    
    <span class="k">print</span> <span class="n">json</span><span class="p">.</span><span class="nf">dumps</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="nf">json</span><span class="p">(),</span> <span class="n">indent</span><span class="o">=</span><span class="mi">4</span><span class="p">,</span> <span class="n">sort_keys</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>

    <span class="k">return</span> <span class="n">response</span><span class="p">.</span><span class="nf">json</span><span class="p">()</span></code></pre></figure>

<p>Now, I needed <a href="http://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html" target="_blank">to package</a> the code to make it ready to deploy:</p>

<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># 1. create folder</span>
<span class="nb">mkdir </span>blogposter
<span class="nb">cd </span>blogposter

<span class="c"># 2. create and copy the code above into poster.py</span>
<span class="nb">touch </span>poster.py

<span class="c"># 3. install requests package locally</span>
pip <span class="nb">install </span>requests <span class="nt">-t</span> <span class="nb">.</span>

<span class="c"># 4. zip contents of the folder and create a zip file in a parent directory</span>
zip <span class="nt">-r9</span> ../pyposter.zip <span class="k">*</span></code></pre></figure>

<p>And we are ready to deploy to AWS! Start with creating a new Lambda Function and selecting a blank function for this job.</p>

<p><img src="/assets/images/posts-images/2017-01-29-confluence-and-aws_screen1.png" alt="Start with selecting a blueprint" /></p>

<p>I didn&#8217;t choose any triggers for now and continued to the configuration page. When presented with function configuration you need to give it a name, description, runtime, and a .zip file with the code. At the end, you will need to define a handler (which will typically be <code class="language-plaintext highlighter-rouge">filename.function_name</code>), and a role.</p>

<p><img src="/assets/images/posts-images/2017-01-29-confluence-and-aws_screen2.png" alt="Fill out the rest of info" /></p>

<p>At the end of the wizard, you can run it manually by pressing the Test button. If you did all the steps above, this should now create a new post on your Confluence. Congratulations, you have now set up a stand-alone function in the Cloud!</p>

<hr />

<h2 id="schedule-it">Schedule it!</h2>

<p>Now that I have automated the task and deployed it to the Cloud, I needed something to trigger that function. In this case, I wanted to schedule it to run weekly, so function was set to be triggered by the <a href="http://docs.aws.amazon.com/lambda/latest/dg/with-scheduled-events.html" target="_blank">CloudWatch</a> events. It offers a good support for <a href="http://docs.aws.amazon.com/AmazonCloudWatch/latest/events/RunLambdaSchedule.html" target="_blank">scheduling Lambda expressions</a>.</p>

<p>I defined event source as a cron expression, and a target as a Lambda function I created in the previous step.</p>

<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># 9:05, every Sunday</span>
5 9 ? <span class="k">*</span> 1 <span class="k">*</span></code></pre></figure>

<p><img src="/assets/images/posts-images/2017-01-29-confluence-and-aws_screen3.png" alt="Adding a scheduled event" /></p>

<p>Now you (and I) have a function that is scheduled to run weekly and does not require a dedicated machine to run. It also has some good options for logging and sending alarms in case it fails.</p>

<hr />]]></content><author><name>Rustam Mehmandarov</name></author><summary type="html">Automating posting to Confluence with cloud services like Amazon Web Services (AWS).</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://mehmandarov.com/assets/images/posts-images/clouds.jpg"/><category term="blog"/><category term="field notes"/><category term="automation"/><category term="cloud"/><category term="python"/><category term="english"/></entry><entry><title type="html">Five Pillars of a Good Maven Project</title><link href="https://mehmandarov.com/five-pillars-of-a-good-maven-project/" rel="alternate" type="text/html" title="Five Pillars of a Good Maven Project"/><published>2016-07-23T08:26:00+02:00</published><updated>2016-07-23T08:26:00+02:00</updated><id>https://mehmandarov.com/five-pillars-of-a-good-maven-project</id><content type="html" xml:base="https://mehmandarov.com/five-pillars-of-a-good-maven-project/"><![CDATA[<p><em>What makes a Maven project good to work with and easy to maintain? There are five types of Maven plugins that will simplify the development process and increase maintainability of a project.</em></p>

<ul>
  <li><a href="#1-technical-aspects">1. Technical Aspects</a></li>
  <li><a href="#2-legal-aspects">2. Legal Aspects</a></li>
  <li><a href="#3-rapid-development">3. Rapid Development</a></li>
  <li><a href="#4-documentation">4. Documentation</a></li>
  <li><a href="#5-testing-and-qa">5. Testing and QA</a></li>
  <li><a href="#bonus-video">Bonus: Video</a></li>
</ul>

<hr />

<p><a href="https://maven.apache.org/">Apache Maven</a> is a build automation tool used primarily for Java projects. It has been around for a while, and it does not seem to be going anywhere anytime soon (whether some of you like it or not). Recently, it has been confirmed yet again, this time in ZeroTurnaround&#8217;s <a href="http://zeroturnaround.com/rebellabs/java-tools-and-technologies-landscape-2016-trends/">Java Tools and Technologies Landscape Report 2016</a> (see the <em>Build Tools</em> section).</p>

<p>The good thing about Maven is that you can do almost anything with it and its plugins. With such a great ecosystem of plugins, able to do nearly anything, one might wonder where to begin when setting up a new Maven project or improving your old one.</p>

<p>I usually like to think of five main categories &#8211; or pillars &#8211; that we will be looking into in this post. Each category will have a set of example plugins that are meant to serve merely as a starting point.</p>

<p>In this post, I will assume that you have some experience with build tools in general and Maven in particular.</p>

<hr />

<h2 id="1-technical-aspects">1. Technical Aspects</h2>

<p>First of all, you would like to make sure that all the technical stuff is in place. You might want to automate the packaging, the checks if the project is running the latest version of all artifacts, that all the dependencies are met, and that the unused dependencies are removed. You might even want to define your own rules for all the things that cause troubles, or just simply annoy you down the line, making the build fail if they are not met.</p>

<p>With all that automation in place, you will be getting predictable results and nice packaging from the first day of the project.</p>

<p>Some of the plugins that should be mentioned here:</p>

<ul>
  <li><strong><a href="http://www.mojohaus.org/versions-maven-plugin/">Versions</a> plugin</strong> does the versions management of artifacts in a project&#8217;s POM file</li>
  <li><strong><a href="http://maven.apache.org/plugins/maven-dependency-plugin/">Dependency</a> plugin</strong> helps you analyzing dependencies, building dependency trees, showing unused dependencies, etc</li>
  <li><strong><a href="http://maven.apache.org/plugins/maven-assembly-plugin/">Assembly</a> plugin</strong> helps you packaging all the dependencies, modules, site documentation, and other files into a single distributable archive</li>
  <li><strong><a href="http://maven.apache.org/enforcer/maven-enforcer-plugin/">Enforcer</a> plugin</strong> lets you make your own rules! You can set up your build to break if some of your requirements are not met</li>
</ul>

<hr />

<h2 id="2-legal-aspects">2. Legal Aspects</h2>

<p>With all the technical stuff out of the way, we might want to make sure that the legal side is taken care of as well.</p>

<p>Yes, I know, it might be less fun thinking about licenses than writing code, but it is still something that has to be done. You still have to release your code under some kind of license, and you will have to make sure that the third-party licenses do not violate your own licensing. So, why not leave that job to a plugin?</p>

<p>The following plugin will manage the license of a maven project and its dependencies; it will also update file headers, download dependency licenses, check third-party licenses, etc.</p>

<ul>
  <li><strong><a href="http://www.mojohaus.org/license-maven-plugin/">License</a> plugin</strong></li>
</ul>

<hr />

<h2 id="3-rapid-development">3. Rapid Development</h2>

<p>Now, back to coding. Or even better &#8211; to seeing the results of your hard work. The chances are that you will need some kind of web or application server for running your code, and you want to able to deploy to that server in no time. Another kind of plugins that will be helping you from the very first day of the project.</p>

<p>Some of the plugins that can be mentioned here:</p>

<ul>
  <li><strong><a href="http://tomcat.apache.org/maven-plugin.html">Apache Tomcat</a> plugin</strong></li>
  <li><strong><a href="https://mvnrepository.com/artifact/org.eclipse.jetty/jetty-maven-plugin">Jetty</a> plugin</strong></li>
  <li><strong><a href="https://docs.jboss.org/wildfly/plugins/maven/latest/">WildFly</a> plugin</strong></li>
  <li><em>&#8230;or any other deploy plugin, depending on your application</em></li>
</ul>

<hr />

<h2 id="4-documentation">4. Documentation</h2>

<p>Every decent project must also be properly documented. However, this is something that developers might postpone until the end. Well, no more! This kind of plugins will help you to get started early and will help you to produce some beautiful (<em>maybe?</em>) and maintainable docs.</p>

<p>You might even consider coupling these with some rules in the Enforcer plugin, but tread carefully as too many, and too strict rules can, and usually do, backfire.</p>

<ul>
  <li><strong><a href="https://maven.apache.org/plugins/maven-site-plugin/">Site</a> plugin</strong></li>
  <li><strong><a href="http://asciidoctor.org/docs/asciidoctor-maven-plugin/">Asciidoctor</a> plugin</strong></li>
</ul>

<hr />

<h2 id="5-testing-and-qa">5. Testing and QA</h2>

<p>By now, your code should be looking great, with all the right licenses, and up and running in no time. So, how about squashing some [virtual] bugs? The list below might help you setting up a proper QA environment and fixing bugs before the come crawling to your production servers.</p>

<ul>
  <li><strong><a href="http://maven.apache.org/surefire/maven-surefire-plugin/">Surefire</a> plugin</strong></li>
  <li><strong><a href="http://maven.apache.org/surefire/maven-failsafe-plugin/">Failsafe</a> plugin</strong></li>
  <li><strong><a href="http://www.mojohaus.org/sonar-maven-plugin/project-info.html">SonarQube</a> plugin</strong></li>
  <li><strong><a href="http://gleclaire.github.io/findbugs-maven-plugin/">FindBugs</a> plugin</strong></li>
  <li><strong><a href="https://maven.apache.org/plugins/maven-pmd-plugin/">PMD</a> plugin</strong></li>
</ul>

<hr />

<h2 id="bonus-video">Bonus: Video</h2>

<p>A video of a talk I gave about this topic at <a href="http://2015.javazone.no/details.html?talk=86734cc36c24b081d399454534248f3aad7062ce30de5aea27de84f80a476269">JavaZone</a> in Oslo (in Norwegian).</p>

<iframe src="https://player.vimeo.com/video/138955650?byline=0&amp;portrait=0" width="640" height="360" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe>

<hr />]]></content><author><name>Rustam Mehmandarov</name></author><summary type="html">The five main types of Maven plugins that will simplify the development process and increase maintainability of a project.</summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://mehmandarov.com/assets/images/posts-images/desk-pencils_1200x800.jpg"/><category term="blog"/><category term="field notes"/><category term="maven"/><category term="software development"/><category term="build and deploy"/><category term="automation"/><category term="java"/><category term="english"/></entry><entry><title type="html">What Happens to Your Code After a Commit?</title><link href="https://mehmandarov.com/what-happens-to-your-code-after-a-commit/" rel="alternate" type="text/html" title="What Happens to Your Code After a Commit?"/><published>2016-05-21T09:45:00+02:00</published><updated>2016-05-21T09:45:00+02:00</updated><id>https://mehmandarov.com/what-happens-to-your-code-after-a-commit</id><content type="html" xml:base="https://mehmandarov.com/what-happens-to-your-code-after-a-commit/"><![CDATA[<p><em>I have written an article for the student magazine INDEX at the Department of Informatics (Ifi), University of Oslo. It was published on 30.04.2016, in Norwegian, and I have <a href="/hva-skjer-med-koden-din-etter-at-du-har-skrevet-den-ferdig/">previously shared it</a> with you here. I was also asked to share an English version of the article. This post is not only a simple translation to English but rather a 2.0 version of the previous post.</em></p>

<ul>
  <li><a href="#automation">Automation</a></li>
  <li><a href="#automatic-tests-and-code-quality-checks">Automatic tests and code quality checks</a></li>
  <li><a href="#environments">Environments</a></li>
  <li><a href="#versions">Versions</a></li>
  <li><a href="#what-now">What now?</a></li>
</ul>

<p><img src="/assets/images/posts-images/2016-05-16-hva-skjer-med-koden-din etter-at-du-har-skrevet-den-ferdig_article-2016-04-27_full.png" alt="The Screenshot of the Article" /></p>

<hr />

<p>Imagine the following scenario: You are a student and have been asked to write either a brand new program or make changes to the code that already exists. So, how do you proceed? You start writing code, you compile and run it periodically to test that it runs and does what it should. You write some more code, and you run it again. The process goes on until you are satisfied and the code has to be delivered. After that, you package it nicely with some documentation, upload it to a page where it has to be delivered and wait for feedback. Sounds familiar?</p>

<p>The process is not so far away from what you might experience at a workplace &#8211; just in miniature form. You have someone who orders functionality &#8211; in this case, your teacher, but later a customer or product owner &#8211; and you have one or several deliveries to be shipped. From a functional perspective, you will want to be sure of three things: that the ordered functionality is implemented, that the new code does not mess up anything else, and that the code will behave the same way it does for you when the customer runs it on their machines or servers.</p>

<p>This simple &#8220;code-test-code&#8221; method works fine for small tasks, but what would you do if you were working on a large project with several colleagues, or even teams, working on different parts of the codebase? How can you check that your code still works when you have made your changes? How can you be sure that your changes have not messed up others&#8217; code? And, how can you ensure that code that works on your machine still works when you deliver it to the customer?</p>

<p>Welcome to my workday. I often work with large IT systems consisting of hundreds of thousands lines of code and with dozens of developers working on different parts of the code. In addition to being a technical project manager with overall responsibility for the solution including application architecture, build infrastructure, and environments the solution runs on, I have often had the role of delivery manager. The latter is responsible for making sure that the final package that is being delivered contains all the code and functionality. Each delivery must have all the bits included, they must work correctly, and the recipients of the package will have to be able to install the software on their systems without any surprises.</p>

<hr />

<h2 id="automation">Automation</h2>

<p>A process to build, deploy and deliver a system consists of several steps, and most of them can usually be automated.</p>

<p>There are many reasons why you might want to automate this process. One reason is that it should be easier for you, and everyone else, to build the system. Automation shortens the time it takes to build it, making it easier to test a change you made to the codebase.</p>

<p>The second reason is that it makes it easier for others to take over or help on a project without having to familiarize themselves with the entire build process to get started. It will also make it easier to build with different build configurations. In addition to all this, it will also eliminate the human factor in the whole process &#8211; we humans make mistakes, and by automating the manual labor, we minimize the chance of making those errors.</p>

<p>Depending on the task at hand, the automation can be achieved by using various tools. Everything from simple make and shell scripts, to sophisticated build tools, like <em>Apache Maven</em> and <em>Gradle</em>, can help you reach your goals. Most often it will be not one, but a combination of the tools mentioned above. These can help you with both general automation of builds and deploy process, and automate the steps that will be described below.</p>

<p><img src="/assets/images/posts-images/2016-05-16-hva-skjer-med-koden-din etter-at-du-har-skrevet-den-ferdig_ci-superhero.png" alt="CI to the Rescue" /></p>
<figcaption class="caption"> Illustration made after my drawings, and for this article by Mahasty Assi</figcaption>

<hr />

<h2 id="automatic-tests-and-code-quality-checks">Automatic tests and code quality checks</h2>

<p>Once the code is written, and the build can be achieved with a single press of a button or by running a single script, you can begin to look at another important part of automation &#8211; running tests and managing code quality.</p>

<p>The automatic tests can be run both at low-level &#8211; <em>unit tests</em>, and at the functional level. Unit tests check that your methods are working properly, for example, that, given some specific input parameters, they return an expected answer. The <em>functional tests</em> on the other hand, will test the system as a whole, helping you to determine whether your system can still perform a set of specific tasks it was designed to do. There is also a third kind of tests that can, and should, be automated &#8211; <em>integration tests</em>. These are the tests used to check if the system is communicating with other systems in the way it was intended.</p>

<p>A failing test &#8211; no matter type &#8211; can tell you that something dramatic has happened in your system and that the system has ceased to behave as expected, giving you an opportunity to spot potential errors much earlier in the development process.</p>

<p>Another way to discover hidden problems is to run static code analysis that can detect known errors patterns, security weaknesses, bad coding habits, or unnecessary complexity. <em>PMD</em>, <em>FindBugs</em> and <em>SonarQube</em> can be mentioned as an example of tools for code quality management in Java world (while <em>NDepend</em>, <em>StyleCop</em> and <em>SonarQube</em> can be used for .NET projects).</p>

<hr />

<h2 id="environments">Environments</h2>

<p>When developing a system, one should know which environment and setup the system will run on when it goes live, i.e., when it gets deployed to a production environment. This information will help you to set up similar environments for development, and especially for testing. This is an important step in detecting errors that may occur only with a specific set-up or specific hardware. When these environments are set up, you can also automate the deployment of your code. This can be very convenient and save you a lot of time. Imagine, you push a change to a version control system (because you are using a <a href="https://en.wikipedia.org/wiki/Version_control">VCS</a>, right?), and it&#8217;s rolled out in a suitable environment &#8211; ready to be tested or demonstrated in no time, and with no effort from your side.</p>

<hr />

<h2 id="versions">Versions</h2>

<p>Versioning of the code and the applications will make sure you can simply answer two important questions:</p>

<ul>
  <li>Which version of the code is running here?</li>
  <li>What functionality is part of the particular version of the application?</li>
</ul>

<p>Both of these questions are something you are going to ask yourself and will be asked by others. The answers to these will help you when fixing or reporting bugs, or when working on a new functionality that is being rolled out.</p>

<p>If you are creating a larger system, you will also need this information when writing Release Notes for the latest version.</p>

<p>Versions will also help you keep track of and updating documentation for your system.</p>

<hr />

<h2 id="what-now">What now?</h2>

<p>Automation of build and deploy is a complex topic. The points that I mentioned here can be used as a starting point &#8211; a kind of checklist for a better system. You do not need to do everything at once or go in depth on all points. You should rather start with one of them and work your way towards full automation.</p>

<p>These points &#8211; and a few more &#8211; are a part of a continuous delivery process that is designed to ensure that it will only take a few minutes between pushing a change to version control system till it is live in production. You do not need to go that far in the beginning, but it may be something to strive for.</p>

<p>These points may seem tedious, but they will often have an immediate effect on your development process &#8211; they will simplify the entire process of handling a system, so you can spend more time on what you like best, namely programming!</p>

<hr />]]></content><author><name>Rustam Mehmandarov</name></author><summary type="html">I have written an article for the student magazine INDEX at the Department of Informatics (Ifi), University of Oslo. It was published on 30.04.2016, in Norwegian, and I have previously shared it with you here. I was also asked to share an English version of the article. This post is not only a si...</summary><category term="blog"/><category term="automation"/><category term="continuous delivery"/><category term="build and deploy"/><category term="testing"/><category term="english"/></entry><entry><title type="html">Hva skjer med koden din etter at du har skrevet den ferdig?</title><link href="https://mehmandarov.com/hva-skjer-med-koden-din-etter-at-du-har-skrevet-den-ferdig/" rel="alternate" type="text/html" title="Hva skjer med koden din etter at du har skrevet den ferdig?"/><published>2016-05-16T09:45:00+02:00</published><updated>2016-05-16T09:45:00+02:00</updated><id>https://mehmandarov.com/hva-skjer-med-koden-din-etter-at-du-har-skrevet-den-ferdig</id><content type="html" xml:base="https://mehmandarov.com/hva-skjer-med-koden-din-etter-at-du-har-skrevet-den-ferdig/"><![CDATA[<p><em><strong>Update 21.05.2016:</strong> <a href="/what-happens-to-your-code-after-a-commit/">English version</a> of this article.</em></p>

<p><em>Jeg har skrevet en artikkel for studentmagasinet INDEX som utgis ved Institutt for Informatikk (Ifi), Universitetet i Oslo. Den ble publisert 30.04.2016, og er tilgjengelig for alle studenter p&#229; Ifi. N&#229; vil jeg ogs&#229; &#229; dele den med alle dere andre som ikke kunne sikre dere en utgave av INDEX p&#229; fredag. Enjoy!</em></p>

<ul>
  <li><a href="#automatiser">Automatiser</a></li>
  <li><a href="#automatiske-tester-og-sjekk-av-kodekvalitet">Automatiske tester og sjekk av kodekvalitet</a></li>
  <li><a href="#milj&#248;er">Milj&#248;er</a></li>
  <li><a href="#versjoner">Versjoner</a></li>
  <li><a href="#veien-videre">Veien videre</a></li>
</ul>

<p><img src="/assets/images/posts-images/2016-05-16-hva-skjer-med-koden-din etter-at-du-har-skrevet-den-ferdig_article-2016-04-27_full.png" alt="The Screenshot of the Article" /></p>

<hr />

<p>Tenk deg f&#248;lgende scenario: Du er en student og har f&#229;tt i oppgave om &#229; skrive enten et helt nytt program, eller gj&#248;re noen endringer i noe kode som finnes fra f&#248;r av. S&#229;, hvordan g&#229;r du fram? Du begynner &#229; skrive kode, du kompilerer og kj&#248;rer den med jevne mellomrom for &#229; teste at den fungerer og gj&#248;r det den skal. Du skriver mer, du kj&#248;rer den igjen. Slik fortsetter du til du er forn&#248;yd og koden skal leveres. Etter det pakker du koden din pent i en arkiv med litt dokumentasjon, laster den opp p&#229; en side hvor det skal leveres og venter p&#229; tilbakemelding. H&#248;res det kjent ut?</p>

<p>Prosessen er vel ikke s&#229; langt unna det du vil oppleve i arbeidslivet, bare i miniatyrform. Du har noen som bestiller en type funksjonalitet &#8211; i dette tilfelle din l&#230;rer, men senere en kunde eller produkteier &#8211; og du har en eller flere leveranser som skal leveres til den som har bestilt jobben. Fra et funksjonelt perspektiv vil du v&#230;re sikker p&#229; tre ting &#8211; at den bestilte funksjonaliteten er med, at den ikke &#248;delegger for noe annet og at koden vil oppf&#248;re seg p&#229; samme m&#229;te n&#229;r bestilleren kj&#248;rer det hos seg.</p>

<p>Den enkle &#171;kode-teste-kode&#187;-metoden fungerer fint for sm&#229; oppgaver, men hva ville du gjort hvis du skulle jobbe p&#229; et stort prosjekt hvor flere jobber p&#229; forskjellige deler av koden? Hvordan kan du sjekke at koden din fortsatt fungerer n&#229;r du har gjort dine endringer? Hvordan kan du v&#230;re sikker p&#229; at dine endringer ikke har &#248;delagt andres kode?</p>

<p>Og, hvordan kan du sjekke at koden som fungerer p&#229; din maskin fortsatt fungerer n&#229;r du leverer den fra deg?</p>

<p>Velkommen til min arbeidshverdag. Jeg jobber ofte med store datasystemer som best&#229;r av mange hundre tusener kodelinjer og med flere titalls utviklere som jobber med forskjellige deler av den koden. I tillegg til &#229; v&#230;re teknisk prosjektleder med overordnet ansvar for blant annet applikasjonsarkitektur, bygg, og milj&#248;er den kj&#248;rer p&#229;, har jeg samtidig hatt rollen som leveranseansvarlig. Sistnevnte har ansvaret med &#229; til slutt pakke all koden og levere den til kunden. Ved hver leveranse m&#229; man passe p&#229; at alle biter er med, at de fungerer som de skal og at mottakeren av pakken vil kunne installere det p&#229; sitt system uten noen overraskelser.</p>

<hr />

<h2 id="automatiser">Automatiser</h2>

<p>En prosess med &#229; bygge, deploye og levere et system best&#229;r ofte av flere steg og de kan som regel automatiseres. Det er mange &#229;rsaker til at man &#248;nsker automatisering. Den ene &#229;rsaken er at det skal v&#230;re enklere for deg og alle andre &#229; bygge systemet &#8211; det korter ned tiden det tar &#229; gj&#248;re det, ergo &#229; teste en endring du har gjort i koden. Den andre &#229;rsaken er at det kan gj&#248;re det lettere for andre &#229; ta over eller hjelpe til p&#229; et prosjekt uten &#229; m&#229;tte sette seg inn i hele byggeprosessen. Det vil ogs&#229; gj&#248;re det lettere &#229; bygge med forskjellige konfigurasjoner dersom man &#248;nsker det. I tillegg til alt dette, vil det ogs&#229; eliminere den menneskelige faktoren i hele prosessen. Vi mennesker kan gj&#248;re feil, og ved &#229; automatisere manuelt arbeid fjerner vi en kilde for en del feil.</p>

<p>Automatisering kan oppn&#229;s ved hjelp av forskjellige verkt&#248;y. Alt fra enkle make- og shell-skript til ganske avanserte byggeverkt&#248;y som Apache Maven og Gradle kan hjelpe deg, litt avhengig av oppgaven som skal gj&#248;res. Oftest blir det en kombinasjon av noen ovennevnte verkt&#248;y. Disse kan hjelpe deg b&#229;de med generell automatisering av bygge- og deploy-prosessen, og med automatisering av punktene under.</p>

<p><img src="/assets/images/posts-images/2016-05-16-hva-skjer-med-koden-din etter-at-du-har-skrevet-den-ferdig_ci-superhero.png" alt="CI to the Rescue" /></p>
<figcaption class="caption">Illustrasjon laget etter mine tegninger, og for denne artikkelen av Mahasty Assi</figcaption>

<hr />

<h2 id="automatiske-tester-og-sjekk-av-kodekvalitet">Automatiske tester og sjekk av kodekvalitet</h2>

<p>N&#229;r koden din er skrevet og byggingen av den er et knappetrykk eller en skriptkj&#248;ring unna, kan du begynne &#229; se p&#229; en annen viktig del av automatisering &#8211; kj&#248;ring av tester og sjekk av kodekvalitet. De automatiske testene kan v&#230;re b&#229;de p&#229; lavniv&#229; &#8211; enhetstester &#8211; og p&#229; funksjonell niv&#229;. Enhetstester sjekker at metodene dine fungerer som de skal, for eksempel &#229; returnere riktig svar, gitt noen inputparametre. De funksjonelle testene vil teste systemet som en svart boks. Her vil du f&#229; svar om systemet ditt fortsatt kan utf&#248;re et sett med bestemte oppgaver. Det finnes ogs&#229; integrasjonstester som kan og b&#248;r automatiseres, og disse brukes for &#229; sjekke at systemet snakker med og til andre systemer p&#229; den m&#229;ten det var tiltenkt.</p>

<p>Uansett type tester kan disse, dersom de feiler, fortelle deg om at noe dramatisk har skjedd i ditt system og om at systemet har sluttet &#229; oppf&#248;re seg som forventet. Det vil ogs&#229; gi deg en mulighet til &#229; oppdage potensielle feil mye tidligere i utviklingsprosessen.</p>

<p>En annen m&#229;te &#229; oppdage skjulte problemer er &#229; kj&#248;re statisk kodeanalyse som kan oppdage kjente m&#248;nstre for feil, sikkerhetssvakheter, uvaner eller un&#248;dvendig kompleksitet.</p>

<hr />

<h2 id="milj&#248;er">Milj&#248;er</h2>

<p>N&#229;r du utvikler et system b&#248;r du vite hvilket milj&#248; og oppsett dette vil kj&#248;re p&#229; n&#229;r systemet g&#229;r live, dvs. legges ut i produksjonsmilj&#248;et. Denne informasjonen kan du bruke til &#229; sette opp tilsvarende milj&#248;er for utvikling, og spesielt med tanke p&#229; testing. Dette er n&#248;dvendig for &#229; v&#230;re i stand til &#229; oppdage feil som vil kunne oppst&#229; kun med et bestemt oppsett, eller spesifikk hardware. N&#229;r disse milj&#248;ene er satt opp kan du automatisere utrulling av din kode til disse milj&#248;ene. Dette kan v&#230;re veldig praktisk, og spare deg for mye tid. Tenk at du sjekker inn en endring (for du bruker vel et versjonskontrollsystem, ikke sant?), og vips, er det ute p&#229; et passende milj&#248;, klar til &#229; testes eller demonstreres.</p>

<hr />

<h2 id="versjoner">Versjoner</h2>

<p>Versjonering av koden vil gj&#248;re at du lett kan svare p&#229; to viktige sp&#248;rsm&#229;l:</p>

<ul>
  <li>Hvilken versjon av koden kj&#248;rer her?</li>
  <li>Hvilken funksjonalitet er en del av bestemt versjon?</li>
</ul>

<p>Begge disse sp&#248;rsm&#229;lene er noe du kommer til &#229; sp&#248;rre deg selv og vil bli spurt om fra andre. Svarene p&#229; disse vil hjelpe deg n&#229;r du skal jobbe med feil som blir fikset eller oppdaget, eller ny funksjonalitet som blir rullet ut. Dersom du lager et st&#248;rre produkt, vil du ogs&#229; f&#229; bruk for disse n&#229;r du skal lage Release Notes for den nyeste versjonen. Versjoner vil ogs&#229; hjelpe deg &#229; holde styr p&#229; en oppdatert dokumentasjon for ditt system.</p>

<hr />

<h2 id="veien-videre">Veien videre</h2>

<p>Automatisering av bygg og deploy er et veldig stort tema. Punktene som jeg nevner her kan brukes som et springbrett, en slags sjekkliste for et bedre system. Du trenger ikke gj&#248;re alt p&#229; en gang, eller g&#229; i dybden p&#229; alle punkter, men start et sted og jobb deg videre mot full automatisering.</p>

<p>Disse punktene &#8211; og litt mer &#8211; er en del av en kontinuerlig leveranseprosess som skal bidra til at det kun vil ta noen minutter fra du sjekker inn en endring til det er ute i produksjon. S&#229; langt trenger du ikke &#229; g&#229; i f&#248;rste omgang, men det kan jo v&#230;re noe &#229; strekke seg etter? Men det disse punktene kan gj&#248;re med en gang er &#229; forenkle din hverdag s&#229; vel som hele prosessen rundt h&#229;ndtering av et system, slik at du kan bruke mer tid p&#229; det du liker best, nemlig programmering!</p>

<p><em>Denne artikkelen ble publisert i magasinet INDEX (2. utgave, 2016), Institutt for Informatikk, Universitetet i Oslo.</em></p>

<hr />]]></content><author><name>Rustam Mehmandarov</name></author><summary type="html">Update 21.05.2016: English version of this article.</summary><category term="blog"/><category term="automation"/><category term="continuous delivery"/><category term="build and deploy"/><category term="testing"/><category term="norwegian"/></entry></feed>
