18 Jun 2019

In this post, I'm going to describe how we can configure nginx to be able to cache responses based on the specific cookie value. 

Let's imagine a situation when you want to do an A/B test on your website, where 50% of users should see new headline text on the page. Other 50% of visitors will continue see the old page. In this case, all your server-side manipulation about splitting users to different versions will be ignored by nginx cache (of course, if you have it).

It happens because nginx, by default, has this configuration:

proxy_cache_key $scheme$proxy_host$request_uri;

So, it will use the same cache key for all users, who requests page with the same url.

Luckily, nginx allows us easily to customize proxy_cache_key! What we need to do, it's just to add a specific cookie to this key:

proxy_cache_key $scheme$proxy_host$request_uri$cookie_MY_COOKIE_NAME; 

And that's it! After this, if userA has MY_COOKIE_NAME=A and userB has MY_COOKIE_NAME=B they will receive different versions of page by the same url.


If you need more complex behaviour, where you won't use hardcoded cookie name in your nginx config, you can do something like this:

if ($http_cookie ~* "ab_(.*?)=([\w-]+)" ) {
  set $abcookie $1$2;
}

proxy_cache_key $scheme$proxy_host$request_uri$abcookie; 

As you can see, you can use $http_cookie and generate proxy_cache_key based on specific cookie patterns. In this concrete example we check if http cookies contains cookie with regex pattern "ab_(.*?)=([\w-]+)" and if this cookie exists, we generate new variable for proxy_cache_key.


28 May 2019

Currently, we work on migration from REST API to GraphQL for our services. We build a GraphQL server on top of SQL database for that, using gqlgen library.

When we created our schema:

type MenuLink {
  id: ID!
  langcode: String!
  title: String!
  url: String!
  additional: String
}

And generated struct:

type MenuLink struct {
    ID         int            `json:"id"`
    Langcode   string         `json:"langcode"`
    Title      string         `json:"title"`
    URL        string         `json:"link__uri"`
    Additional string         `json:"additional"`
}

we faced error "Scan error on column index 2: unsupported Scan, storing driver.Value type <nil> into type *string", when we tried to extract data from SQL database table and assign it to the struct. This happened because "additional" column contains NULL values sometimes and scanner is unable to convert null values into string.

To solve this issue, we just have to use "sql.NullString" instead of "string". But how we can tell to gqlgen to generate structs with this type?

The answer is not so complicated.

First, we have to add a new custom scalar (let's name it as NullableString) to graphql schema and change String type for additional attribute to it:

type MenuLink {
  id: ID!
  langcode: String!
  title: String!
  url: String!
  additional: NullableString
}

scalar NullableString

After that we have build an external marshaler for this new type:

package scalars

import (
    "database/sql"

    "github.com/99designs/gqlgen/graphql"
)
// MarshalNullString returns graphql.Null if string is NULL. If not, then uses default string implementation. 
func MarshalNullString(ns sql.NullString) graphql.Marshaler {
    if !ns.Valid {
        return graphql.Null
    }
    return graphql.MarshalString(ns.String)
}


func UnmarshalNullString(v interface{}) (sql.NullString, error) {
    if v == nil {
        return sql.NullString{Valid: false, String: ""}, nil
    }

    s, err := graphql.UnmarshalString(v)
    return sql.NullString{String: s, Valid: err == nil}, err
}


Here I just created a new package with name scalars where we can keep our marshalers.

And then in .gqlgen.yml we have to point to our marshaler:

schema:
- schema.graphql
exec:
  filename: generated.go
model:
  filename: models/generated.gopackage: models
struct_tag: json
models:
  NullableString:
    model: gitlab.mfb.io/user/graphql_server/scalars.NullString
resolver:
  filename: resolver.gotype: Resolver

That's it! After that if you run

$ go run github.com/99designs/gqlgen

It will generate struct with *sql.NullString type:

type MenuLink struct {
    ID         int             `json:"id"`
    Langcode   string          `json:"langcode"`
    Title      string          `json:"title"`
    URL        string          `json:"link__uri"`
    Additional *sql.NullString `json:"additional"`
}


22 May 2019

As a person who came to Go world from PHP, I had some troubles when I tried to iterate over previously generated map, because of

assuming that the order of items will be the same with an order with was used during map generation. 

Let's check an example:

package main


import (
    "fmt"
)


func main() {
    // Generates items map with 10 elements.
    items := make(map[int]string)
    for i := 0; i < 10; i++ {
        items[i] = fmt.Sprintf("This is item %d", i)
    }
    // Processes items from map.
    for _, item := range items {
        performItem(item)
    }
}


func performItem(item string) {
    fmt.Println(item)
}

The result of execution will be:

This is item 5
This is item 7
This is item 0
This is item 3
This is item 4
This is item 8
This is item 9
This is item 1
This is item 2
This is item 6

After second execution:

This is item 7
This is item 8
This is item 9
This is item 2
This is item 5
This is item 6
This is item 4
This is item 0
This is item 1
This is item 3

As you can see, order of items completely random and does not match with order what we used during generation. This was a little bit strange.

But then I found a fair answer why this is happening:

Programmers had begun to rely on the stable iteration order of early versions of Go, which varied between implementations, leading to portability bugs. If you require a stable iteration order you must maintain a separate data structure that specifies that order.

So, what we can do to process items accordingly to our order?

We just have to sort our keys separately and iterate over our map, using sorted keys:

package main

import (
    "fmt"
    "sort"
)

func main() {
    items := make(map[int]string)
    for i := 0; i < 10; i++ {
        items[i] = fmt.Sprintf("This is item %d", i)
    }

    // Generates keys map.
    keys := make([]int, len(items))
    for k := range items {
        keys[k] = k
    }
    sort.Ints(keys)
    // Iterates over items map, using sorted keys.
    for _, k := range keys {
        performItem(items[k])
    }
}

func performItem(item string) {
    fmt.Println(item)
}

Let's execute it:

This is item 0
This is item 1
This is item 2
This is item 3
This is item 4
This is item 5
This is item 6
This is item 7
This is item 8
This is item 9

Now it looks perfect!


Source code you can found here

15 May 2019

In this post I'm going to describe how can we limit user access to the specific url in golang web application. I will use chi router - a lightweight, idiomatic and composable router for building Go HTTP services.

Let's create our main package.

package main

import (
	"net/http"
	"github.com/go-chi/chi"
)

func main() {
  r := chi.NewRouter()

  r.Get("/", homePageHandler)
  r.Get("/admin", adminPageHandler)
  http.ListenAndServe(":3000", r)
}

func homePageHandler(w http.ResponseWriter, r *http.Request) {
  w.Write([]byte("This is home page")) 
}

func adminPageHandler(w http.ResponseWriter, r *http.Request) {
  w.Write([]byte("This is admin page")) 
}

After this, if we go to the /admin page, we will see this:

Now, let's make this path accessible only for admin.

We have to replace

r.Get("/admin", adminPageHandler)

With

r.Mount("/admin", adminRouter())

Mount attaches another http.Handler or chi Router as a subrouter along a routing path.

Then, we have to attach middleware inside adminRouter() function.

func adminRouter() http.Handler {
    r := chi.NewRouter()
    // Middleware with access rules for router.
    r.Use(AdminOnly)
    r.Get("/", adminPageHandler)

    return r
}

In this middleware we have a simple check is user authorized to access this page or not.

func AdminOnly(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        // If user is admin, allows access.
        if IsLoggedInAdmin(r) {
            next.ServeHTTP(w, r)
        } else {
            // Otherwise, 403.
            http.Error(w, http.StatusText(403), 403)
            return
        }

        return
    })
}

In sake of demonstration, I'm going just to use a random bool function to decide is used admin or not. You can modify this function according to your user authentication model.

func IsLoggedInAdmin(r *http.Request) bool {
    return rand.Float32() < 0.5
}

And that's it. Looks really simple, Isn't it?

Let's go to to the /admin page again.

As you see, now, sometimes (depends on our random decider), user has no access to this page anymore.


You can find source code here

07 May 2019

Testing of functions with database interactions always was challenging. Recently, I found a wonderful library go-sqlmock which will simplify writing tests and mocking database queries in golang applications a lot. And I want to share a short example of how to work with it.

First, we have to install it

go get github.com/DATA-DOG/go-sqlmock

 We have this function with SQL query:

package menu

func MenuByNameAndLanguage(ctx context.Context, db *sql.DB, name string, langcode string) (*models.Menu, error) {
    result, err := db.Query("SELECT id, langcode, title, link__uri, view_sidemenu FROM menu_link_content_data WHERE menu_name=? AND langcode=?",
         name,
         langcode,
    )
    defer result.Close()
    
    var menuLinks []models.MenuLink
    for result.Next() {
         menuLink := models.MenuLink{}
         err = result.Scan(&menuLink.ID, &menuLink.Langcode, &menuLink.Title, &menuLink.URL, &menuLink.SideMenu)
         menuLinks = append(menuLinks, menuLink)    
    }
    menu := &models.Menu{
         Name: name,
         Links: menuLinks,
    }
    return menu, err
}


This function just getting menu links by menu name and language. 

And now let's test it.

We are going to test that MenuByNameAndLanguage function will return correct Menu structure.


package menu


import (
    "context"
    "testing"


    "github.com/DATA-DOG/go-sqlmock"
    "github.com/stretchr/testify/assert"
    "gitlab.mfb.io/KingCron/graphql_server/models"
)


func TestShouldReturnCorrectMenu(t *testing.T) {

    // Creates sqlmock database connection and a mock to manage expectations.
    db, mock, err := sqlmock.New()

    if err != nil {
        t.Fatalf("an error '%s' was not expected when opening a stub database connection", err)
    }
    // Closes the database and prevents new queries from starting.
    defer db.Close()

    // Here we are creating rows in our mocked database.
    rows := sqlmock.NewRows([]string{"id", "langcode", "title", "link__uri", "view_sidemenu"}).
        AddRow(1, "en", "enTitle", "/en-link", "0").
        AddRow(2, "en", "enTitle2", "/en-link2", "0")

    // This is most important part in our test. Here, literally, we are altering SQL query from MenuByNameAndLanguage
    // function and replacing result with our expected result. 
    mock.ExpectQuery("^SELECT (.+) FROM menu_link_content_data*").
        WithArgs("main", "en").
        WillReturnRows(rows)


    ctx := context.TODO()

    // Calls MenuByNameAndLanguage with mocked database connection in arguments list. 
    menu, err := MenuByNameAndLanguage(ctx, db, "main", "en")

    // Here we just construction our expecting result.
    var menuLinks []models.MenuLink
    menuLink1 := models.MenuLink{
        ID:       1,
        Title:    "enTitle",
        Langcode: "en",
        URL:      "/en-link",
        SideMenu: "0",
    }
    menuLinks = append(menuLinks, menuLink1)

    menuLink2 := models.MenuLink{
        ID:       2,
        Title:    "enTitle2",
        Langcode: "en",
        URL:      "/en-link2",
        SideMenu: "0",
    }

    menuLinks = append(menuLinks, menuLink2)

    expectedMenu := &models.Menu{
        Name:  "main",
        Links: menuLinks,
    }
    
    // And, finally, let's check if result from MenuByNameAndLanguage equal with expected result.// Here I used Testify library (https://github.com/stretchr/testify).
    assert.Equal(t, expectedMenu, menu)
} 

 

As you see everything in this example was pretty straightforward.

For more details, you can refer to GoDocs.