How to automate uploading of Android applications to Amazon

<h1>How to automate build and testing in Flutter with Gitlab</h1>
Let&#39;s use Docker image by <a href="https://github.com/PlugFox/docker_flutter">Plugfox</a>, which is targeting Flutter and comes with all dependencies required, such as Android SDK, JVM and so on. 
In this example, we&#39;ll be using free tier of Gitlab, and in order to minify the load on the flow, we&#39;ll define the GIT_DEPTH variable, which should speed up the 
cloning of repository alongside other environment variables, which are required during the building of <a href="https://developer.android.com/guide/app-bundle">APK/AppBundle</a>.
<h2>Prerequisites for this pipeline</h2>
<ol>
<li>Have a <a href="https://developer.android.com/studio/publish/app-signing#generate-key">android keystore</a> file. You can quickly generate it using following command:</li>
</ol>
<pre><code class="hljs language-bash">keytool -genkey -v -keystore example.keystore -alias alias_name -keyalg RSA -keysize 2048 -validity 10000
</code></pre><ol start="2">
<li>Setup your <a href="https://docs.gitlab.com/ee/ci/secure_files/">Gitlab secure files</a>.
</li>
<li>Prepare integration test, for example
</li>
</ol>
<pre><code class="hljs language-dart">import &#x27;package:flutter_test/flutter_test.dart&#x27;;
import &#x27;package:integration_test/integration_test.dart&#x27;;

void main() {
 IntegrationTestWidgetsFlutterBinding.ensureInitialized();

 group(&#x27;example test&#x27;, () {
 testWidgets(&#x27;dummy test&#x27;, (tester) async {
 expect(10, greaterThanOrEqualTo(2));
 });
 });
}
</code></pre><h2>Building release appbundle</h2>
In order to build the Flutter application, we will use the keystore, which you configured at the prerequisite 1 and upload it to 
the Gitlab secure files. In order to download secure files, Gitlab requires to execute shell script, let&#39;s run it: 
<pre><code class="hljs language-bash">curl --silent &quot;https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/download-secure-files/-/raw/main/installer&quot; | bash
</code></pre>Now, let&#39;s configure the signing process. In order to do so, we will use key.properties file. Make sure, that your build.gradle inside flutter 
application. Check the <code>android/app/build.gradle</code> file, and make sure it has something like this:
<pre><code class="hljs language-gradle">def keystoreProperties = new Properties()
def keystorePropertiesFile = rootProject.file(&#x27;key.properties&#x27;)
if (keystorePropertiesFile.exists()) {
 keystoreProperties.load(new FileInputStream(keystorePropertiesFile))
}

android {
 // rest of configuration...

 signingConfigs {
 release {
 keyAlias keystoreProperties[&#x27;keyAlias&#x27;]
 keyPassword keystoreProperties[&#x27;keyPassword&#x27;]
 storeFile keystoreProperties[&#x27;storeFile&#x27;] ? file(keystoreProperties[&#x27;storeFile&#x27;]) : null
 storePassword keystoreProperties[&#x27;storePassword&#x27;]
 }
 }
}
</code></pre>Now, let&#39;s prepopulate the key.properties file with necessary details for signing to work
<pre><code class="hljs language-bash">echo &quot;storeFile=$(pwd)/key.keystore&quot; &gt; android/key.properties
echo &quot;storePassword=$KEYSTORE_PASSWORD&quot; &gt;&gt; android/key.properties
echo &quot;keyAlias=$ALIAS&quot; &gt;&gt; android/key.properties ## Testing Flutter application in CI system
echo &quot;keyPassword=$KEY_PASSWORD&quot; &gt;&gt; android/key.properties
</code></pre>Finally, you can execute building command like this:
<pre><code class="hljs language-bash">flutter build appbundle --no-pub --obfuscate --split-debug-info=debug-info-appbundle # if you need appbundle format
flutter build apk --no-pub --obfuscate --split-debug-info=debug-info-apk # if you need apk
</code></pre><h2>Implementing CI testing in Flutter</h2>
In order to test, make sure that your Android device is visible with the <code>adb devices</code> command. The following steps assume that you have followed this step. 
Optional 
If you would like to precache the Android SDK, you can use following command for extracting the <code>compileSdk</code> value and pass it to sdkmanager for installation
<pre><code class="hljs language-bash">cat android/app/build.gradle | grep compileSdk | tr -d -c 0-9 | /opt/android/cmdline-tools/latest/bin/sdkmanager &quot;platforms;android-$(cat -)&quot;
</code></pre>Usually, tests assume that your screen is unlocked, so we can schedule unlocking of your screen beforehands by inputting following command:
<pre><code class="hljs language-bash">adb shell input keyevent 26 &amp;&amp; adb shell input touchscreen swipe 930 880 930 380
</code></pre>Optional 
If you want to have the screen opened upon installation of your application, you can modify your <code>MainActivity.kt</code> with the following changes: 
<pre><code class="hljs language-kotlin">override fun onCreate(savedInstanceState: Bundle?) {
 super.onCreate(savedInstanceState);
 if (BuildConfig.DEBUG) { // we don&#x27;t want to have this behavior on release builds, so keep it only to tests
 val keyguardManager = getSystemService(Context.KEYGUARD_SERVICE) as KeyguardManager;
 if (Build.VERSION.SDK_INT &gt;= Build.VERSION_CODES.P) {
 setShowWhenLocked(true)
 setTurnScreenOn(true)
 keyguardManager.requestDismissKeyguard(this, null)
 } else {
 // For older versions of Android, you can use the deprecated method as a fallback
 keyguardLock?.disableKeyguard()
 window.addFlags(WindowManager.LayoutParams.FLAG_TURN_SCREEN_ON or WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
 }
 }
}
</code></pre>We also were having timeouts while connecting to the device and Flutter VM. We solved it using following command: 
<pre><code class="hljs language-bash">sed -i &#x27;/).timeout(/,/);/{ s/).timeout.*/);/; t; d;}&#x27; /opt/flutter/packages/flutter_tools/lib/src/test/integration_test_device.dart
</code></pre>Finally, execute <code>flutter test path/to/your_test.dart</code> and you&#39;re ready to go!
<h2>Complete .gitlab-ci.yml</h2>
Here&#39;s the complete .gitlab-ci.yml for building and testing of Flutter applications
<pre><code class="hljs language-yaml">image: plugfox/flutter:3.10.6-android

variables:
 GIT_DEPTH: &quot;1&quot;
 KEYSTORE_PASSWORD: your_keystore_password
 ALIAS: your_alias
 KEY_PASSWORD: your_key_password

stages:
 - build
 - test
 - deploy

e2e_test:
 stage: test 
 interruptible: true
 only:
 - merge_requests
 script: 
 - flutter pub get
 - flutter precache --no-web
 - cat android/app/build.gradle | grep compileSdk | tr -d -c 0-9 | /opt/android/cmdline-tools/latest/bin/sdkmanager &quot;platforms;android-$(cat -)&quot;
 - flutter build apk --no-pub --debug
 - adb shell input keyevent 26 &amp;&amp; adb shell input touchscreen swipe 930 880 930 380
 - sed -i &#x27;/).timeout(/,/);/{ s/).timeout.*/);/; t; d;}&#x27; /opt/flutter/packages/flutter_tools/lib/src/test/integration_test_device.dart
 - flutter test integration_test/app_test.dart

build_flutter_bundle:
 stage: build
 timeout: 2 hours
 variables:
 SECURE_FILES_DOWNLOAD_PATH: &#x27;./&#x27;
 script:
 - curl --silent &quot;https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/download-secure-files/-/raw/main/installer&quot; | bash
 - echo &quot;storeFile=$(pwd)/key.keystore&quot; &gt; android/key.properties
 - echo &quot;storePassword=$KEYSTORE_PASSWORD&quot; &gt;&gt; android/key.properties
 - echo &quot;keyAlias=$ALIAS&quot; &gt;&gt; android/key.properties
 - echo &quot;keyPassword=$KEY_PASSWORD&quot; &gt;&gt; android/key.properties
 - flutter pub get --verbose
 - flutter precache --no-web
 - cat android/app/build.gradle | grep compileSdk | tr -d -c 0-9 | /opt/android/cmdline-tools/latest/bin/sdkmanager &quot;platforms;android-$(cat -)&quot;
 - apk add openjdk17-jre-headless openjdk17-jdk # you should install the java version that your application was written with during CI
 - flutter build appbundle --no-pub --obfuscate --split-debug-info=debug-info-appbundle
 - flutter build apk --no-pub --obfuscate --split-debug-info=debug-info-apk
 - cp build/app/outputs/bundle/app-release.aab ./app-release.aab
 - cp build/app/outputs/flutter-apk/app-release.apk ./amazon-release.apk
 artifacts:
 paths:
 - app-release.aab
 - app-release.apk
 expire_in: 1 week
 rules:
 - if: $CI_COMMIT_BRANCH == &#x27;master&#x27;
 when: manual
 - when: never
</code></pre>Photo by <a href="https://unsplash.com/@toluobde?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Tolu Olubode</a> on <a href="https://unsplash.com/photos/PlBsJ5MybGc?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>

How to automate building and testing of Flutter app

Autopublish TechDocs to Confluence

<h1>Automating Ionic mobile applicaton building</h1>
Today for developing a cross-platform application, you have a lot of options, like <a href="https://flutter.dev">Flutter</a>, <a href="https://cordova.apache.org/">Cordova</a>, <a href="https://ionicframework.com/">Ionic</a>, <a href="https://reactnative.dev/">React Native</a> and so on. 
When comes to development, frameworks/solutions, which utilize JavaScript are quite popular, due to the smoother transition from Web to Mobile. 
In this article let&#39;s focus on Ionic framework, which is framework-agnostic and can be developed with Vanilla JS. 
Namely, we will be looking at <a href="https://capacitorjs.com/">Capacitor</a>, which is recommended by Ionic team.
<h2>How to create Vanilla Ionic project</h2>
As been mentioned, Ionic have possibility to build with such libraries/frameworks as React/Angular, but it does not limiting in choice of frameworks. 
Let&#39;s create a folder <code>vanilla-ionic-app</code> and install Ionic dependencies. In this example we&#39;ll use yarn, but you can utilize <code>npm</code> as well.
We will also use Webpack for better 
<pre><code class="hljs language-bash">mkdir vanilla-ionic-app
cd vanilla-ionic-app
npm init -y 
yarn add -D @capacitor/core @capacitor/cli @capacitor/android webpack webpack-cli html-webpack-plugin
npx cap init 
npx cap add android
</code></pre>This should create a basic skeleton, where you should be able to write smallest example for building application
Open your preferred text editor and make following changes: 
<ol>
<li>Add src/index.js file and fill it with following:</li>
</ol>
<pre><code class="hljs language-javascript">function App() {
 // let&#x27;s show to user simple greeting
 const helloWorld = document.createElement(&#x27;h1&#x27;);
 helloWorld.innerHTML = &#x27;Hello World!&#x27;;
 document.body.appendChild(helloWorld);
}
App();
</code></pre><ol start="2">
<li>Make sure, that your capacitor.config.json looks similar to this:</li>
</ol>
<pre><code class="hljs language-json">{
 &quot;appId&quot;: &quot;com.example.app&quot;,
 &quot;appName&quot;: &quot;vanilla-ionic-app&quot;,
 &quot;webDir&quot;: &quot;build&quot;,
 &quot;bundledWebRuntime&quot;: false
}
</code></pre><h2>Creating the necessary steps in Gitlab pipeline</h2>
For compilation of Capacitor application you should have both NodeJS and Java installed on the system. 
For that reason, we have constructed a <a href="https://github.com/FallenAngel97/alpine-android-nodejs">base Docker image</a>, which we will use in this example.
In order to compile the Ionic/Capactitor application you need to execute the build step for Webpack, copy result to the <code>android</code> folder and compile using Gradle
For reference, you can use snippet below and adjust for your needs
<pre><code class="hljs language-yaml">default:
 image: ghcr.io/fallenangel97/alpine-android-nodejs:master

stages:
- build

build_android:
 stage: build
 script:
 - yarn
 - yarn build
 - npx cap sync android
 - npx cap copy
 - cd android
 - ./gradlew bundleRelease
 - cp ./app/build/outputs/bundle/release/app-release.aab ../app-release.aab

 artifacts:
 paths:
 - ./app-release.aab
 expire_in: 1 week
</code></pre>Photo by <a href="https://unsplash.com/@dovilerm?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Dovile Ramoskaite</a> on <a href="https://unsplash.com/s/photos/vanilla?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>

How to automate building your Ionic application on gitlab

<h1>How to auto deploy from Gitlab to VPS with Docker Swarm</h1>
When working with the dynamically changing websites, it&#39;s crucical to have
own CI/CD process. This way you can speed up the delivery of the product from
development to the end customers. Nowadays, multiple Git hosting services are
providing custom CI/CD pipelines, which can be extended as you like. For this 
post, let&#39;s take <a href="https://gitlab.com">Gitlab</a> as an example of hosting provider.
<h2>Docker swarm configuration</h2>
<a href="https://docs.docker.com/engine/swarm/">Docker swarm</a>, in short, is a way how 
to orchestrate your docker containers. Within the swarm stack you can define 
decentralization and optimize your scaling. For this tutorial, let&#39;s define 
some sample swarm by executing following commands:
<pre><code class="hljs language-bash">docker swarm init --advertise-addr &lt;your_ip_address&gt; # this command outputs swarm join command
docker swarm join --token &lt;token&gt; &lt;ip_address&gt; # replace it with the command above
</code></pre>Next, we will need to create a docker compose file, which can be updated from
the pipeline. 
You can use following example as the starting point: 
<pre><code class="hljs language-yaml">version: &#x27;3.7&#x27;
services:
 # ... other services
 node_container:
 image: $DOCKER_IMAGE # necessary for dynamic substitution of the image
 ports:
 - &quot;3000:3000&quot; # modify it with your own ports which the application is listening to
 deploy: # necessary to provide the behaviour for updating the stack
 update_config:
 order: start-first
 failure_action: rollback
 delay: 10s
 rollback_config:
 parallelism: 0
 order: stop-first
 restart_policy:
 condition: any
 delay: 5s
 max_attempts: 3
 window: 120s 
</code></pre>With the minimal setup above, let&#39;s try to spin up the server with the following command: 
<pre><code class="hljs language-bash">docker stack deploy -c /path/to/compose --with-registry-auth gitlab-automation-example
</code></pre><h2>Gitlab runner</h2>
After successfully finishing previous section, you should already have a spined up 
compose with the microservices. 
Let&#39;s proceed with this process by installing the Gitlab runner and configuring it on 
your VPS. 
You should follow Gitlab documentation for installation of the Gitlab runner <a href="https://docs.gitlab.com/runner/install/linux-repository.html">here</a>
But in case you are using Ubuntu/Debian, you can just copy-paste following commands:
<pre><code class="hljs language-bash">curl -L &quot;https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh&quot; | sudo bash
sudo apt-get install gitlab-runner
</code></pre>After this, go to your project settings and copy-paste token and URL for your use case
<img src="/blog/example-of-gitlab-tokens.png" alt="Example of Gitlab tokens">
<pre><code class="hljs language-bash">sudo gitlab-runner register -n \
 --url https://gitlab.com/ \
 --registration-token REGISTRATION_TOKEN \
 --executor shell \
 --description &quot;My Runner&quot;
sudo usermod -aG docker gitlab-runner
</code></pre>If you will reload page on Gitlab, you should see, that your runner should be listed under 
&quot;Available specific runners&quot;
<img src="/blog/added-runner-gitlab.png" alt="Example of specific runner">
<h2>.gitlab-ci.yml</h2>
The final step to compile your project is to construct the .gitlab-ci.yml. 
The complete guide is located <a href="https://docs.gitlab.com/ee/ci/yaml/gitlab_ci_yaml.html">here</a>, but you can read 
example here and follow it to construct your own. 
<pre><code class="hljs language-yaml">image: docker:20.10.16

variables:
 GIT_DEPTH: 1 # To keep the .git folder small enough, let&#x27;s limit shallow to 1 https://docs.gitlab.com/ee/ci/pipelines/settings.html#limit-the-number-of-changes-fetched-during-clone

stages: # this instructions indicates the amount and order of steps in your pipeline
 - build
 - deploy

build-docker-image:
 stage: build
 variables:
 IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
 script:
 - echo &quot;Building Docker image $IMAGE_TAG ...&quot;
 - echo &quot;Credentials $CI_REGISTRY_USER - $CI_REGISTRY_PASSWORD ${CI_REGISTRY}&quot;
 - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD ${CI_REGISTRY}
 - docker build -t $IMAGE_TAG .
 - docker push $IMAGE_TAG
 - docker system prune --force

deploy:
 stage: deploy
 variables:
 IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
 script:
 - echo &quot;Updating the container...&quot;
 - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD ${CI_REGISTRY} # In order to fetch the necessary image, you should login to gitlab registry beforehand
 - DOCKER_IMAGE=$IMAGE_TAG docker stack deploy -c /home/gitlab-runner/deploy/compose.yml --with-registry-auth gitlab-automation-example # replace &quot;gitlab-automation-example&quot; with your own service created in previous steps
 - docker container prune --force
 - docker image prune --force # this ensures that after building of the image, we clean everything after ourselves
</code></pre>If you have followed tutorial step by step, in the end you should be having following picture with your pipeline configuration: 
<img src="/blog/example-of-pipeline.png" alt="Example of pipeline">
Photo by Lukas: <a href="https://www.pexels.com/photo/boats-near-dock-296242/">https://www.pexels.com/photo/boats-near-dock-296242/</a>

How to deploy from Gitlab to VPS 2022

<h1>How to generate DOCX from Markdown</h1>
Markdown is a format, which is usually used for communication and providing documentation
for developers and technical products. Markdown supports a lot of necessary features,
which help with organizing your documentation. 
Yet, for sharing the documents, usually used DOCX standard, which can be edited
via specialized software, despite markdown can be edited in any text editor.
For converting documents, we can use pandoc utility, which can achieve exactly 
desired behavior by converting from one format to another.
Pandoc can be extended by Lua filters and tweaked per our needs. 
<pre><code class="hljs language-lua">local cacheLinks = {} -- Stores the found local links in the repository

function file_exists(name)
 local f=io.open(name,&quot;r&quot;)
 if f~=nil then io.close(f) return true else return false end
end

function Link (link, target, title, attr)
 local linkTarget = link.target -- where the link points to
 if file_exists(linkTarget) then
 print(&quot;Found the link: &quot; .. linkTarget)
 table.insert(cacheLinks, linkTarget)
 return {}, {}, {}, {} -- return empty table, which will remove Link from the flow of the document
 end
 return link
end

function Pandoc(blocks, meta)
 local paragraphs = {}
 for i = 1, #cacheLinks do -- iterate over the local links table
 local fh = io.open(cacheLinks[i]) -- read the contents of that linked file
 local contents = pandoc.read( -- parse it with pandoc with markdown parser
 fh:read &#x27;*a&#x27;,
 &quot;markdown&quot;,
 PANDOC_READER_OPTIONS
 ).blocks
 blocks.blocks:extend(contents) -- append parsed data to the end of the document
 end
 return blocks, meta
end
</code></pre>Then, inside your pipeline, you should call the following:
<pre><code class="hljs language-bash">pandoc --lua-filter fix-links-multiple-files.lua -o output.docx -f markdown -t docx README.md --toc
</code></pre>Photo by Kindel Media: <a href="https://www.pexels.com/photo/a-close-up-shot-of-paper-clipped-documents-7054757/">https://www.pexels.com/photo/a-close-up-shot-of-paper-clipped-documents-7054757/</a>