Tag: private cloud

  • Nextcloud cron job

    I switched to system cron for my nextcloud cron jobs. Using docker, this is my cron job command:

    */5  *  *  *  *    /usr/bin/docker exec -u 33 srv_nc_1 /bin/bash -c 'php -f /var/www/html/cron.php'
    
  • Nextcloud out-of-memory PHP error

    I got this error when upgrading to nextcloud 21:

    PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 20480 bytes) [...]
    

    Turns out this is the way to solve it: Prefix PHP_MEMORY_LIMIT=512M to the commands. In combination with docker it looks like this:

    /usr/bin/docker exec -u 33 srv_nc_1 /bin/bash -c 'PHP_MEMORY_LIMIT=512M php occ db:add-missing-indices'
    
    ... OR USING -e ...
    
    /usr/bin/docker exec -u 33 -e PHP_MEMORY_LIMIT=512M srv_nc_1 /bin/bash -c 'php occ db:add-missing-indices'
    
    ... OR MANUALLY ...
    
    ~$ docker exec -tiu 33 srv_nc_1 /bin/bash
    ~# PHP_MEMORY_LIMIT=512M php occ db:add-missing-indices
    

    Source: https://is.gd/mua5wv

  • nextcloud and Docker and reverse proxies

    I have a nextcloud setup like described here (docker-compose, let’s encrypt proxy companion, postgres and nextcloud). And for a while I couldn’t connect any new nextcloud clients to the installation.

    This fixed it:

    <?php
    $CONFIG = array (
      # manually added because it's not picked up from
      # the env vars once set ... it seems ...
    
      # the docker IP range
      'trusted_proxies' => ["172.16.0.0/12"],
    
      # the hostname of the server
      'overwritehost'   => "my.super.secret.server",
    
      # the ENDUSER->PROXY protocol, NOT the proxy-> nextcloud protocol!
      'overwriteprotocol' => "https",
    
      #
      # AAAND NOW back to the original config file ...
      # ...
    )
    

    Some notes:

  • Databases with dokku

    This is part 2 of a couple of blog posts about dokku, a amazing little Heroku clone.

    In the previous post I showed how to set up Dokku on a DigitalOcean droplet, and deployed a little hello-world container with a single git push. The reason why I wanted dokku thoug was the need of a database. As said – hosting comes cheap, databases usually come either expensive or with limited flexibility, or just too annoying configuration effort.

    Dokku is the perferct middle ground. Let’s see why.

    For me it was the existing postgres plugin which you can simply install and use. The whole process is incredibly easy, takes wbout two commands, and looks like this (let’s assume our “hello world” container uses a database):

    $ sudo dokku plugin:install https://github.com/dokku/dokku-postgres.git postgres
    

    That’s it, again.

    $ dokku postgres:create hello-world
    
    dokku postgres:create hello-world
    Waiting for container to be ready
    Creating container database Securing connection to database
    =====> Postgres container created: hello-world
    =====> Container Information
           Config dir: /var/lib/dokku/services/postgres/hello-world/config
           Data dir: /var/lib/dokku/services/postgres/hello-world/data
           Dsn: postgres://postgres:bd6b0725d710bb5a662bb628eee787b1@dokku-postgres-hello-world:5432/hello_world
           Exposed ports: -
           Id: 785ef252c748ed85739d1d6ad375a1e1bd66e925ac79358e9ffaa30ab852d6c0
           Internal ip: 172.17.0.9
           Links: -
           Service root: /var/lib/dokku/services/postgres/hello-world
           Status: running
           Version: postgres:10.2
    
    $ docker ps
    
    CONTAINER ID   IMAGE                      COMMAND                  CREATED         STATUS         PORTS      NAMES
    cc99cccacf2c   dokku/hello-world:latest   "/bin/sh -c 'php-fpm…"   2 minutes ago   Up 2 minutes   80/tcp     hello-world.web.1
    785ef252c748   postgres:10.2              "docker-entrypoint.s…"   5 minutes ago   Up 5 minutes   5432/tcp   dokku.postgres.hello-world
    [...]
    

    This creates a database container with postgres 10.2, as you can see. You can influence a lot of behavior by using environment variables, see the GitHub page for more info.

    Then you link the container to the running app:

    $ dokku postgres:link hello-world hello-world
    -----> Setting config vars
           DATABASE_URL: postgres://postgres:bd6b0725d710bb5a662bb628eee787b1@dokku-postgres-hello-world:5432/hello_world
    -----> Restarting app hello-world
    -----> Releasing hello-world (dokku/hello-world:latest)...
    -----> Deploying hello-world (dokku/hello-world:latest)...
    -----> Attempting to run scripts.dokku.predeploy from app.json (if defined)
    -----> No Procfile found in app image
    -----> DOKKU_SCALE file found (/home/dokku/hello-world/DOKKU_SCALE)
    =====> web=1
    -----> Attempting pre-flight checks
           For more efficient zero downtime deployments, create a file CHECKS.
           See http://dokku.viewdocs.io/dokku/deployment/zero-downtime-deploys/ for examples
           CHECKS file not found in container: Running simple container check...
    -----> Waiting for 10 seconds ...
    -----> Default container check successful!
    -----> Running post-deploy
    -----> Configuring hello-world.my-paas.for-myself.com...(using built-in template)
    -----> Creating http nginx.conf
    -----> Running nginx-pre-reload Reloading nginx
    -----> Setting config vars DOKKU_APP_RESTORE: 1
    -----> Found previous container(s) (14c349cb496d) named hello-world.web.1
    =====> Renaming container (14c349cb496d) hello-world.web.1 to hello-world.web.1.1535285386
    =====> Renaming container (cc99cccacf2c) serene_bassi to hello-world.web.1
    -----> Attempting to run scripts.dokku.postdeploy from app.json (if defined)
    -----> Shutting down old containers in 60 seconds
    =====> 14c349cb496d95cc4be1833f2e7f6ef2bef099a37c2a22cd4dcdb542f09bea0f
    =====> Application deployed:
           http://hello-world.my-paas.for-myself.com
    

    And done.

    What happened? You have now the environment variable $DATABASE_URL set in the hello-world app, that’s why the restart was necessary (which you can postpone, if you want, but you probably need it now, right?).

    Let’s check:

    $ docker exec -ti hello-world.web.1 /bin/sh
    
    [now in the container]
    
    # env | grep DATABASE
    DATABASE_URL=postgres://postgres:bd6b0725d710bb5a662bb628eee787b1@dokku-postgres-hello-world:5432/hello_world
    

    That’s it. Super easy. Now if you’re using Django, you could use kennethreitz/dj-database-url to automatically parse and use it, and you’re done. (Probably every framework has something similar, so just have a look).

  • OpenStack IPs, part II

    Just a short one. Now that I can list IPs (see last post), it might be nice to associate them on the command line, too. First I wanted to extend the little script, but then I remembered there must already be a CLI way for this.

    And there is.

    $ openstack server list
    $ nova floating-ip-associate <server-uuid> <floating-IP>
    

    (Note: The floating-ip is the actual IP, not the UUID of the OpenStack floating IP)

    Groovy.

  • OpenStack floating IP convenience

    Problem: I am working in a tenant which has a couple of hosts with floating IPs assigned. I always have to look them up either manually using the command line clients (and dealing with all those UUIDs), or manually in the web GUI. Didn’t like.

    Solution: Python script, which outputs FLOATING_IP -> HOST_NAME.

    Here it is.

    #!/usr/bin/env python
    
    from novaclient import client
    import novaclient.v2.floating_ips as os_fips
    import novaclient.v2.servers as os_servers
    import novaclient.v2.networks as os_networks
    
    #from pprint import pprint as pp
    import os
    from sys import exit
    
    
    def error(printme):
        print("ERROR: {}".format(printme))
        exit(-1)
    
    def check_env():
        for a in ("OS_TENANT_ID", "OS_TENANT_NAME", "OS_AUTH_URL", "OS_USERNAME", "OS_PASSWORD"):
            if not os.environ.get(a): error("Please set ${}".format(a))
    
    def get_client():
        check_env()
        return client.Client(2,
                             os.environ["OS_USERNAME"],
                             os.environ["OS_PASSWORD"],
                             os.environ["OS_TENANT_NAME"],
                             os.environ["OS_AUTH_URL"])
    
    
    if __name__=="__main__":
        nova = get_client()
        fipman = os_fips.FloatingIPManager(nova)
        servman = os_servers.ServerManager(nova)
        netman = os_networks.NetworkManager(nova)
    
        ips = fipman.list()
        srs = servman.list()
    
        id2server = {}
        for a in srs:
            id2server[a.id] = a
    
        ips = [(ip.ip, ip.instance_id) for ip in ips]
        # filter out unused floating ips (which have  as instance id)
        ips = filter(lambda x: x[1], ips)
        # create (IP, SERVER_NAME pairs)
        ips = map(lambda x: (x[0], id2server[x[1]].name), ips)
        # sort for convenience by host instance name
        ips = sorted(ips, key=lambda x: x[1].lower())
    
        for a in ips:
            print("{:18s} {}".format(a[0], a[1]))
    

    Sample output:

    $ tools/list_ips
    10.64.108.163      ab_1695_1_dml
    10.64.108.199      ab_1695_1_jump
    10.64.108.160      media0
    10.64.109.97       RJD1_CouchbaseA1
    10.64.109.99       RJD1_JumpServer
    10.64.109.98       RJD1_OpenshiftMaster
    10.64.109.148      tla-centos66
    

    Groovy.


    Main sources: