Currently Browsing: Misc Tech

Docker for Mac fix for max files, transparent hugepages warnings

To avoid Redis and other apps’ “max files limit” and “transparent hugepages” etc errors in Docker containers on Docker for Mac, you can set sysctls options in your docker-compose file by using compose format 2.1 or greater, which should be supported in docker-compose >= 1.10.

in docker-compose.yml file:

version: '2.1'
    image: redis:3.2-alpine
    command: redis-server /usr/local/etc/redis/redis.conf
      - net.core.somaxconn=1024

To update transparent hugepages settings in Docker for Mac’s Alpine hypervisor host system, you can use this hack:

docker run --rm --privileged -ti alpine /bin/sh -c "echo never > /sys/kernel/mm/transparent_hugepage/enabled && echo never > /sys/kernel/mm/transparent_hugepage/defrag"

In theory you wouldn’t need/want to do this on a production host as you would configure the host properly. If you update Docker, you will need to re-run this.

Docker-in-docker Docker Compose with sshd


# Docker client with docker-compose && sshd
# use on a Docker host to allow you to ssh and access Docker and Compose remotely
# e.g., as part of CI/CD on a private network.
# ** Not for production use on publicly-exposed server **
# mount for docker host socket:
#    -v /var/run/docker.sock:/var/run/docker.sock:ro
# mount for docker-compose access (optional):
#    -v /host/compose/root:/opt/compose/alias
# cd or reference -f /alias/to/docker-compose.yml file when using docker-compose ...
FROM docker:17
# (uses Alpine)
RUN apk add --update py-pip
RUN pip install docker-compose
# RUN apk add ca-certificates curl openssl nano
RUN apk add openssh
# use fresh keys: (could also do on startup)
RUN rm -rf /etc/ssh/ssh_host_rsa_key /etc/ssh/ssh_host_dsa_key
RUN /usr/bin/ssh-keygen -A
# install (or append) to authorized_keys: (optional)
COPY certs/ /root/.ssh/authorized_keys
# cleanup install:
RUN rm  -rf /tmp/* /var/cache/apk/*
# remove prior entrypoint if there is one:
CMD ["/usr/sbin/sshd","-D"]

Build ID from Git (updated)

A simple build_id can be generated from Git via:

git log --pretty=oneline | wc -l

which will give you something like 9682

And while this will likely vary by branch, you really want to base a build id only from the branch you are deploying to production (in our case, release) anyway. It won’t let you name a work-in-progress, so that might be an issue for some people but also might enforce good practice by not letting you name a release until it’s “done”. As long as the naming is consistent, it should work. Of course in this scenario if you hot/bug-fix release that actual value will go up, but that’s not really what we care about, seems like our criterion would be:

  • something we can generate in an automated build
  • something sequential so it’s easy to tell the order to other builds
  • something unique (potential edge cases it may not be, but would be very edge-case)

    If you want a sequential number to give ordinal context but hash info for relevancy can use:

    echo $(git log --pretty=oneline | wc -l)-$(git log -1 '--pretty=format:%h')

    which will give you something like 9682-36a51c8

    log count + “-” + truncated last commit hash. You could add zero-padding or similar if desired.

    This seems particularly useful if you aren’t tagging at the time you create the id. Otherwise, you would be able to get the hash from the tag.

  • Zip contents of current directory for Lambda

    To create a AWS Lambda bundle, you need create a zip archive of your app directory that doesn’t contain a parent directory. To do this, cd to your app directory and execute:

    zip -r -X "../" *

    Override Docker Entrypoint properly

    It’s not always so obvious what format the `ENTRYPOINT` override should take in a `docker run` command, so here is an example to clarify if you wanted to get a file list instead of the default entrypoint: first override the default ENTRYPOINT command with `/bin/ls` as a `run` argument then pass any args after the image name.

    docker run --entrypoint "/bin/ls" namespace/imagename -al /container/path/tolist

    As mentioned in the Docker docs, ENTRYPOINT only specifies the executable to run when the container starts.

    « Previous Entries