Andrii's blog Blog about programming

My X Vim Tips

Intro

This document is useful for using Vim (neovim) and other tools.

1. Edit the current shell command in Vim

Note: assuming

export EDITOR=nvim
# Enable Ctrl-x-e to edit the command line
autoload -U edit-command-line
Ctrl-x Ctrl-e

2. Rename a few files at a time using VIM

curl https://raw.githubusercontent.com/thameera/vimv/master/vimv > /usr/local/bin/vimv && chmod +755 /usr/local/bin/vimv

vimv *.txt

P.S. Of course, you need the $EDITOR env variable.

3. Jump to interesting elements in an editor:

$ git jump diff

So assume you have such a diff:

diff --git foo.c foo.c
index a655540..5a59044 100644
--- foo.c
+++ foo.c
@@ -1,3 +1,3 @@
 int main(void) {
-  printf("hello word!\n");
+  printf("hello world!\n");
 }

$ git jump diff

Will open an $EDITOR on foo.c:3.

To be continued…

Go 2 and Numeric Literals

Tl;dr Go version 2 is going to add some changes that will affect numeric literals. Everybody already knew it didn’t they? Let’s see what is already in the go repo in the master branch:

$ git clone git@github.com:golang/go.git
$ cd go/src && ./all.bash
$ go version devel +eee07a8e68 Wed Aug 21 15:20:00 2019 +0000 darwin/amd64

One of the most noticeable for me is of course _ in numbers).

here is example:

package main

func main() {
println("1_200_000 -> ", 1_200_000)
println("3.1415_9265 -> ", 3.1415_9265)
println("0b0110_1101_1101_0010 -> ",0b0110_1101_1101_0010)
// println("0___ -> ", 0___) invalid example from discussion
// println(" 0__.0__e-0__ -> ",0__.0__e-0__) invalid example from discussion
// println("1__2", 1__2) invalid
}

So:

$ go version
$ go1.12.7 darwin/amd64
$ go run main.go
# command-line-arguments
./main.go:4:28: syntax error: unexpected _200_000, expecting comma or )
./main.go:5:35: syntax error: unexpected _9265, expecting comma or )
./main.go:6:39: syntax error: unexpected b0110_1101_1101_0010, expecting comma or )

Oops 😬

Second attempt:

./go version
devel +eee07a8e68 Wed Aug 21 15:20:00 2019 +0000 darwin/amd64
./go run main.go
1_200_000 -> 1200000
3.1415_9265 -> +3.141593e+000
0b0110_1101_1101_0010 -> 28114


Go 1.13RC1:
go get golang.org/dl/go1.13rc1
go1.13rc1 run main.go
1_200_000 -> 1200000
3.1415_9265 -> +3.141593e+000
0b0110_1101_1101_0010 -> 28114

🎉🎉🎉🎉

P.S. It’s not entirely clear why it seems to be the way it should be in Go2, but got into Go 1.13 … I probably missed something, I’ll go read GIthub …

[1] https://go.googlesource.com/proposal/+/master/design/19308-number-literals.md

Debugging Golang Tests With Delve

How to debug golang unit-tests using delve 🐛🔥?

So, often you need to run 1 test and even in debug mode, for example, when you wrote a test that repeats a bug. It’s very simple (although not very obvious from the docks):

dlv test -- -test.run NameOfYourTest/PartOfTheName* 

similar to go test -run.

Or a live example:

➜ debug_test dlv test -- -test.run TestFibonacciBig
(dlv) b main_test.go:6
Breakpoint 1 set at 0x115887f for github.com/andriisoldatenko/debug_test.TestFibonacciBig() ./main_test.go:6
(dlv) c
> github.com/andriisoldatenko/debug_test.TestFibonacciBig() ./main_test.go:6 (hits goroutine(17):1 total:1) (PC: 0x115887f)
1: package main
2:
3: import "testing"
4:
5: func TestFibonacciBig(t *testing.T) {
=> 6: var want int64 = 55
7: got := FibonacciBig(10)
8: if got.Int64() != want {
9: t.Errorf("Invalid Fibonacci value for N: %d, got: %d, want: %d", 10, got.Int64(), want)
10: }
11: }
(dlv)

Alternatively, run with -v (remember go test -v):

➜ debug_test dlv test -- -test.v -test.run TestFibonacciBig
(dlv) c
=== RUN TestFibonacciBig
--- PASS: TestFibonacciBig (0.00s)
PASS

Debugging a Go Application Inside a Multi Stage Docker Container

Let’s imagine that you have a microservice on sale, each of which lives in its own Dockerfile and naturally, like all adult uncles, this is a multi-stage Dockerfile. More details about multi-stage can be found in the docks (https://docs.docker.com/develop/develop-images/multistage-build/), if laziness is just a Dockerfile which has 2 FROM keywords and in we that copy it from one layer to another.

So let’s go 🐎:

docker build -t goapp:latest .
Sending build context to Docker daemon 22.53kB
Step 1/7 : FROM golang AS builder
...
Successfully tagged goapp:latest

Every day we need to develop and debug this application: an attentive reader might suggest installing a debugger inside one of the layers. So let’s add something like:

--- a/Dockerfile
+++ b/Dockerfile
@@ -1,4 +1,5 @@
FROM golang AS builder
+RUN go get -u github.com/go-delve/delve/cmd/dlv
ADD . /src
RUN cd /src && go build -o goapp

And then we run something like this:

➜ debug-multi-stage-docker-and-go docker run -it --entrypoint=dlv goapp
docker: Error response from daemon: OCI runtime create failed: container_linux.go:344: starting container process caused "exec: \"dlv\": executable file not found in $PATH": unknown.

And here’s the bad luck, the same story will be if you have docker-compose for local development or something else. How to be) Docker build has a wonderful –target flag!

docker build --target builder -t goapp:latest .
docker run -it goapp sh
# dlv debug main.go
could not launch process: fork/exec /src/__debug_bin: operation not permitted

And how to fix this ☝️, read my last post https://t.me/golang_for_two/20)

And Voila! A huge plus ⚡️ of this approach for me personally, the lack of Dockerfile-dev and options.

P.S. If you have docker-compose, then in your docker-compose.override.yml you can write like this:

version: "3.4"
services:
app:
image: goapp:dev
build:
context: .
dockerfile: Dockerfile
target: builder

🎉💥🎉

P.S. final gist ->:

Debugging a Go Application Inside a Docker Container

So we need: straight arms and basin with docker preinstalled:

$ cat Dockerfile
FROM golang:1.13

WORKDIR /go/src/app
COPY . .

RUN go get -u github.com/go-delve/delve/cmd/dlv

CMD ["app"]

$ docker build -t my-golang-app .

This is just one of the options, sometimes you need to start dlv instead of bash and so on.

$ docker run -it --rm my-golang-app bash

$ root@03c1977b1063:/go/src/app# dlv main.go
Error: unknown command "main.go" for "dlv"
Run 'dlv --help' for usage.
root@03c1977b1063:/go/src/app# dlv debug main.go
could not launch process: fork/exec /go/src/app/__debug_bin: operation not permitted

OOps…

So let’s add the parameters:

$ docker run -it --rm --security-opt="apparmor=unconfined" --cap-add=SYS_PTRACE my-golang-app bash

voila 🎉

$ root@7dc3a7e8b3fc:/go/src/app# dlv debug main.go
Type 'help' for list of commands.
(dlv)

P.S. same trick can be used with docker-compose or multi-stage builds. If you are interested in how to debug multi-stage builds on Go, please put “+” in the comments or throw a tomato 🍅.

A collection of sport programming 'must reads'

How to Practice

Online Contests:

Advent of code Hackerrank Spoj Codechef Valladolid OJ (http://acm.uva.es/p) Official ACM Live Archive (http://cii-judge.baylor.edu/) Peking University Online Judge (http://acm.pku.edu.cn/JudgeOnline/) Programming Challenges (http://www.programming-challenges.com)

Books

Materials & articles

Configure delve debugger to print long strings

Today I came across the fact that print in debugger mode does not show long lines.

> main.main() ./main.go:7 (PC: 0x10b08d4)
     2:
     3:  import "fmt"
     4:
     5:  func main() {
     6:    v1 := "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
=>   7:    fmt.Println(v1)
     8:  }
(dlv) p v1
"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...+2 more"

Let’s change max-string-len:

(dlv) config -list
aliases            map
substitute-path    
max-string-len     <not defined>
max-array-values   <not defined>
show-location-expr false
(dlv) config max-string-len 1000
(dlv) p v1
"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"

✅ Now we are good to go.

Debugging Golang With Delve

How to debug golang code

Yes, the Go code can and should be debugged. I often clash in different teams, as developers, so far in 2019! debug with prints :)

Delve is the debugger that I usually use every day. If you open the documentation, it’s not perfect to say the least.

You can install it:

go get -u github.com/go-delve/delve/cmd/dlv

Most often I use dlv debug :

dlv debug github.com/andriisoldatenko/go-blog

or

dlv debug main.go
(dlv) breakpoint main.go:1
(dlv) continue

After you have set the breakpoint, you can click continue and the program will stop where needed, and so on. There are also shortcuts (b, c, l and so on).

Origins of term "Serverless"

Serverless is new trend in software development. It’s confusing many developers around the world, let’s try to find origins of term “Serverless”. First time Ken Fromm in 2012 use this term in his article. (Ken added a bit of clarification in the comments to give credit where credit is due.)

Later Badri Janakiraman mentioned that he also heard about usage of the term in 2012 in context of Continuous Integration. For example Travis CI, where you can delegate Continuous Integration and testing of your project to Travis CI.

Later in 2014 Amazon Web Services announced a new service Lambda, a stateless event-driven compute service for dynamic applications that doesn’t require provisioning of any compute infrastructure. As a result in 2015 we see lot’s usage of term serverless.

In October 2015 there was a talk at Amazon’s re:Invent conference titled “The Serverless Company using AWS Lambda”, referring to PlayOn! Sports, where we can see real production usage of AWS Lambda and Serverless architecture. Let’s try to define term “Serverless”. Besides another trends in software development, there is no one definition of this term.

Serverless is delegating hardware or infrastructure to third parties or vendors. Serverless describes applications that depend on services ‘in the cloud’ to manage server-side logic and state. For example we have ‘rich client’
and we want to use the vast ecosystem of cloud accessible databases (like Parse, Firebase), authentication services (Auth0, AWS Cognito), etc. These types of services have been previously described as ‘(Mobile) Backend as a Service’, and I’ll be using ‘BaaS’ as a shorthand in the rest of this article.

Wikipidea said, that Serverless is applications where some amount of server-side logic is still written by the application developer but unlike traditional architectures is run in stateless compute containers that are event-triggered, ephemeral (may only last for one invocation), and fully managed by a 3rd party. (Thanks to ThoughtWorks for their definition in their most recent Tech Radar.)

One way to think of this is ‘Functions as a service / FaaS’ . AWS Lambda is one of the most popular implementations of FaaS at present, but there are others. I’ll be using ‘FaaS’ as a shorthand for this meaning of Serverless throughout the rest of this article. from https://martinfowler.com/articles/serverless.html

More info you can find in my latest talk from Pycon Italia 2017 Building Serverless applications with Python

Eloquent fibonacci sequences in python

“If by chance I have omitted anything more or less proper or necessary, I beg forgiveness, since there is no one who is without fault and circumspect in all matters.”

― Leonardo Bonacci - italian mathematician

Leonardo Bonacci

Quick introduction

The idea of this article to collect eloquent python patterns using well-known Fibonacci sequence.

Recursive approach

cat fibonacci1.py
def fibonacci1(n):
    if n < 2:
        return n
    return fibonacci1(n - 2) + fibonacci1(n - 1)
    
if __name__ == '__main__':
    from timeit import timeit
    print(timeit("fibonacci1(5)", setup="from __main__ import fibonacci1")) 
$ python3 fibonacci1_bench.py # On my MacBook Pro (Mid 2015) 2.5 GHz Intel Core i7, 16 GB 1600 MHz DDR3
2.0335577040023054

Recursive approach using caching

My favorite advice: you should know you language standard library. You can find lot’s of info under functools module docs. Default maxsize is 128 for lru_cache decorator.

$ cat fibonacci2.py
import functools


@functools.lru_cache()
def fibonacci2(n):
    if n < 2:
        return n
    return fibonacci2(n - 2) + fibonacci2(n - 1)

    
if __name__ == '__main__':
    from timeit import timeit
    print(timeit("fibonacci2(5)", setup="from __main__ import fibonacci2"))
$ python3 fibonacci2_bench.py # On my MacBook Pro (Mid 2015) 2.5 GHz Intel Core i7, 16 GB 1600 MHz DDR3
0.09731649799505249
$ cat fibonacci_generator.py

Generator approach for using yield

def fibonacci_generator():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

if __name__ == '__main__':
    from timeit import timeit
    print(timeit("list(itertools.islice(fibonacci_generator(), 5))",
                 setup="from __main__ import fibonacci_generator"))
$ python3 fibonacci_generator.py # On my MacBook Pro (Mid 2015) 2.5 GHz Intel Core i7, 16 GB 1600 MHz DDR3
1.1730475709991879

About timeit instead of conclusion.

Also keep in mind:

Note By default, timeit() temporarily turns off garbage collection during the timing. The advantage of this approach is that it makes independent timings more comparable. This disadvantage is that GC may be an important component of the performance of the function being measured. If so, GC can be re-enabled as the first statement in the setup string.

More info about timeit Lib/timeit.py