Tag: azure

  • GitLab, App Service & CI/CD – Variant 1

    [Updates]

    • 2020-11-29 – reference terraform simple-appservice module
    • 2020-11-29 – add small fix for GitLab CI script

    Since I spent way more time fighting with this I though I might write a proper recap and maybe help others to get started. I also set up a repository with example code which should (!) be a “pull-and-run” thing once you have your ARM_ACCESS_KEY ready, which goes along with this blog post.

    Now, let’s get started. What we want to do here is basically pretty simple, and detailed in the picture below:

    git flow variant 1

    This sounds easy, right? It is, if you are done fighting Azure stupidity and know a couple of things. So what we need to do for this? We need to …

    • Create the App Service (I use terraform for this, and you can use my example repo or my terraform simple-appservice module to get started)
    • Create a GitLab repo (of course, that’s what this is all about, right? An example “hello world” flask up can also be found in the mentioned repo)
    • Configure the GitLab CI system to contain some credentials for the App Service
    • Add a .gitlab-ci.yml file to your GitLab repo to enable CI
    • … done.

    That should be exactly all. Let’s get started.

    Create the app service

    As said – please use my example repo and terraform. That should be enough:

    • make init
    • make plan
    • make do

    Terraform should just output the needed credentials.

    terraform credential output

    Important to know:

    • use at least “B1 | Basic” for your SKU settings
    • your “source_control” setting should be “LocalGit” I guess. There is an error in either Terraform or Azure (I think the latter) if you use “ExternalGit”, which would be “variant 2” (a blog post to follow)

    Create a GitLab repo

    This one should do, and if you are rightfully annoyed by Azure give Render a try, they look cool (just found them about 2 mintes ago). To fork the repo you have to click the very unobtrusive “fork” button in the top right.

    Configure GitLab CI

    The only thing to do is set two environment variables for the test runner. The information you need can be found either from my terraform output, or fom the Azure portal (images below). Then you add two variables in GitLab CI (GitLab repo -> settings -> CI/CD -> Variables):

    • AZ_APP_NAME (in this case this would be “flypenguin-coolapp-xrp”)
    • AP_APP_REPO_PASS (in this case this would be “SYut….”)

    set gitlab variables

    There are again a few caveats:

    • Basically the whole thing is based on the app name. Your user name should be your app name prefixed by a “$” sign, which is highly annoying.
    • If you ever want to use a “$” sign in any GitLab runner environment variable, you have to escape it with another “$”. Example: “my$variable” should be “my**$$**variable” in the “value” field of GitLab.

    Add .gitlab-ci.yml file to repo

    Well, just add the file and push the repo.

    stages:
      - push_to_azure
    push_it:
      stage: push_to_azure
      only:
        - master
      allow_failure: false
      before_script:
        - git config --global user.email "some@email.address"
        - git config --global user.name "GitLab CI Pipeline"
      script:
        - export REPO_FQDN="$AZ_APP_NAME.scm.azurewebsites.net"
        - export REPO_URL="https://\$$AZ_APP_NAME:$AZ_APP_REPO_PASS@$REPO_FQDN/$AZ_APP_NAME.git"
        - git remote add azure_app_service "$REPO_URL"
        - git remote -v
        # DEEP DIVE INFORMATION, DELETE ME IF YOU WANT
        # the local branch is 'detached head' - which is fucked.
        # we can't do "git push --force azure HEAD:master" on the FIRST push.
        # we can't push into an empty repository at all, even when using
        #   git push --force azure HEAD:refs/heads/master
        # because on the first push this will still not work.
        # the actually easiest way seems to be to not do "detached head" here.
        # so let's try to "unshallow" that thing.
        # https://is.gd/t7y7LM / https://is.gd/8YE4Ua
        # https://is.gd/rZDwxy / https://is.gd/rZDwxy
        - git fetch --unshallow origin || true
        - git push --force azure_app_service HEAD:refs/heads/master
    

    Finally – test.

    Just push a change to your repo and see if it works. The first obvious change is the addition of the .gitlab-ci.yml file … .

    Hope that helped, hope it works 🙂

  • Azure App Service & Python, Part II

    God I hate Microsoft.

    TL;DR – use “Basic” and “B1” as your app service SKUs. Anything “below” won’t work and you won’t know why.

    So, why I am again angry? Cause I want to deploy an app service, using terraform. Sounds soooooooooo simple, right? Except I fail at the creation of the app service (even without code, yet).

    Example:

    resource "azurerm_app_service_plan" "plan" {
      name                = "pkd-appservices0"
      location            = azurerm_resource_group.rg.location
      resource_group_name = azurerm_resource_group.rg.name
      kind                = "Linux"
      reserved            = true
    
      sku {
        tier = "Dynamic"     # WRONG, use at least "Basic"
        size = "Y1"          # WRONG, use at least "B1"
      }
    }
    
    resource "azurerm_app_service" "appservice" {
      name                = random_string.server.result
      location            = azurerm_resource_group.rg.location
      resource_group_name = azurerm_resource_group.rg.name
      app_service_plan_id = azurerm_app_service_plan.plan.id
    
      site_config {
        scm_type         = "LocalGit"
        linux_fx_version = "PYTHON|3.8"
      }
    }
    

    Now. What happens? Let’s see:

    Error creating App Service "wzkytawt" (Resource Group "pkd-it-appservices0"):
       web.AppsClient#CreateOrUpdate: Failure sending request: StatusCode=0 --
       Original Error: autorest/azure: Service returned an error. Status=<nil> <nil>
    

    What? Okay, our app service fails without any indication. By setting “TF_LOG=TRACE” and reading through 100s of lines of code we get this:

    Consumption pricing tier cannot be used for regular web apps
    

    What the fuck is a “consumption pricing tier”? Well, turns out MS doesn’t know as well, at least it’s described exactly nowhere. Also, can you get a list of tiers? No! Of course not! Why would you?  Stackoverflow to the rescue, again, btw, and no idea where that guy pulled that priceless list from.

    So, set things to “F1 | Free” and everything works.

    … or not. Now we get this:

    There was a conflict.                                              (really?!)
    
    64 Bit worker processes cannot be used for the site as the plan
    does not allow it.                                                 (WTF?)
    
    For more information on pricing and features, please see:
    https://aka.ms/appservicepricingdetails                            (oh! a link! NICE)
    

    Formatting and comments by me … . Now, wondering why the fuck a multi-billion-dollar corp can do less than my Raspi (run 64bit code) I head over to said page. And find nothing. There is not a single mention about 64bit, 32bit, or anything helpful. Just … prices.

    Assuming the fuckers just want more money I upgrade SKU to “B1 | Basic”, and everything works. (Update: I just saw that this is actually mentioned in the terraform docs 🙂

    This time for real, and I think I might be onto the path of finally understanding everything Microsoft does.

  • Azure App Service & Python, Part I

    A.k.a. “you’ve got to be fucking kidding me”. Azure App Service sucks, as basically any part of Azure. What’s surprising this time is that terraform actually contributes to the sucking on this occasion – usually the Hashi folks do a very good job at reducing the annoyance. But maybe they’ve given up, I wouldn’t blame them.

    So what happened?

    What confused the sh*t out of me the last days is that I see tutorials (all really, really, really bad) that do use Python versions newer than 3.4 (not surprisingly). But, alas, there’s this: the site_config.python_version field of terraform’s azurerm_app_service resource. It allows the values “2.7” and “3.4” (a Python version which eol’ed in March ’19).

    After a shit ton of googling I found this, though. Turns out, it’s not even the right field to use!! To select the Python runtime you should use linux_fx_version, which nobody, nowhere, ever documented. And naive me just assumed that something with “python” in its name would be the right thing to use. Stupid, I know, we’re talking Azure here …

  • The 6 ways of returning data from an Azure Function

    Why have one if you can have many? Well, beats me, but I thought I’d collect them here so I have my personal reference. Disclaimer: This is only for JavaScript based functions (most of the examples on the MS pages are C#).

    So, most of them have to do with the file function.json  file:

    {
      "bindings": [
        {
          "type": "httpTrigger",
          "name": "req",
          "and": "so on ..."
        },
        {
          "type": "http",
          "name": "thisIsAnOutputBinding",
          "direction": "out"
        },
        {
          "type": "someOtherType",
          "name": "anotherBinding",
          "direction": "out"
        }
      ]
    }
    

    More precisely, with the “out” type binding that is defined in the 2nd “bindings” object. The “out” binding has a “name” property, which is basically relevant for all of the methods.

    !!!!! WARNING !!!!! – It seems the name of the binding property must be camelCase**. At least I consistently get “invalid binding” errors when I use snake_case**

    Method 1 – context property

    The context object has one property per defined “out” binding. So if our property is named “thisIsAnOutputBinding”, our result property is context.thisIsAnOutputBinding , or as MicroSoft puts it:

    Outputs (bindings of direction === "out" ) can be written to by a function in a number of ways. In all cases, the name property of the binding as defined in function.json corresponds to the name of the object member written to in your function.

    Simply put, you just assign the value to a context  member and you’re done:

    module.exports = function (context, req) {
      context.thisIsAnOutputBinding = {
        my: "return value",
        "in this case": "an object",
      };
      // contect.done() is only needed for synchronous functions context.done();
    };
    

    Source here.

    Method 2 – return an object (async functions only)

    With the bindings given above, you can assign the values to the context property, but you can also return an object whose keys correspond to the binding names:

    module.exports = async function (context) {
      return { thisIsAnOutputBinding: 42, anotherBinding: 43 };
      // of course, instead of 42/43 you can put any js object here.
    };
    

    Source here.

    Method 3 – using context.bindings

    This is basically a variant of method 1. Why does it exist? No one knows. Apparently the context object has a .bindings  property, which in turn again has properties which name-match the defined out bindings. So this is another possibility, and I think the return  is unnecessary:

    module.exports = async function (context) {
      let retMsg = "Hello, world!";
      context.bindings.thisIsAnOutputBinding = {
        body: retMsg,
      };
      context.bindings.anotherBinding = retMsg;
      return;
    };
    

    Source here.

    Method 4 – using context.done()

    If you’re using a sync method, you can’t return an object, but you can call context.done(err, obj) .

    module.exports = async function (context) {
      context.done(null, {
        thisIsAnOutputBinding: { text: "hello there, world", anotherBinding: 42 },
      });
    };
    

    Source here.

    Method 5 – just return it already (only async functions)

    Probably someone said “Well, if I only have one output binding, why should I explicitly address it?”, which is a valid thought. So another way to return data was invented.

    For this the configuration in function.json looks a bit different:

    {
      "bindings": [
        { "type": "someType", "name": "req", "direction": "in" },
        { "type": "someOutType", "direction": "out", "name": "$return" }
      ]
    }
    

    You have to set exactly one “out” type binding, and the name must be “$return”. Then you can do this:

    module.exports = async function (context, req) {
      return "woohooo!!";
    };
    

    Source here.

    Methods 6 – special for http outputs

    You thought we were done? Noooooo. For http methods, you can diretly return the body object which is then used to create the body

    // actually NO IDEA if this works sync, async or both
    // my guess is: sync, because "context.done() is implicitly called"
    module.exports = async function (context, req) {
      rv = { body: "<html/>", status: 201 };
      context.res.send(rv);
    };
    

    Source here.

    Summary

    Let’s just say it’s a mess, and too many ways ruin the map.

    It works though.