Concurrent Map Writing and Reading in Go, or how to deal with the data races.
16 Jul 2021

Concurrent Map Writing and Reading in Go, or how to deal with the data races.

This time, I will show you how to work with the maps in go effectively and prevent the occurrence of the data race errors. Data races happen when several goroutines access the same resource concurrently and at least one of the accesses is a write.

Let’s write a simple program, which generates a map of numbers and print them to the console:

package main

import "log"

var numbers = make(map[int]int, 100)

func main() {
	generateNumbersMap()
}

func generateNumbersMap() {
    // Write.
	for i := 0; i < 100; i++ {
		numbers[i] = i
	}
    // Read.
	for i := 0; i < 100; i++ {
		log.Print(numbers[i])
	}
}

Now, if we run it with the data race detector option go run -race main.go, we can see the printed list of numbers in the console without any data race problems.

Everything seems to be good. Is it? Let’s add some concurrency to our super complex program 😄 and see what happens:


package main

import (
	"log"
	"sync"
)

var numbers = make(map[int]int, 100)

func main() {
	generateNumbersMap()
}

func generateNumbersMap() {
	wg := sync.WaitGroup{}
    // Write.
	for i := 0; i < 100; i++ {
		wg.Add(1)
		go func(i int) {
			defer wg.Done()
			numbers[i] = i
		}(i)
	}
    // Read.
	for i := 0; i < 100; i++ {
		wg.Add(1)
		go func(i int) {
			defer wg.Done()
			log.Print(numbers[i])
		}(i)
	}

	wg.Wait()
}

If we run it now, in the console we can notice the data race errors:

==================
WARNING: DATA RACE
Write at 0x00c0001241b0 by goroutine 8:
  runtime.mapassign_fast64()
      /usr/local/opt/go/libexec/src/runtime/map_fast64.go:92 +0x0
  main.generateNumbersMap.func1()
      /dev/webdevstation/blog-examples/concurent-map/main.go:68 +0xa4

Previous write at 0x00c0001241b0 by goroutine 7:
  runtime.mapassign_fast64()
      /usr/local/opt/go/libexec/src/runtime/map_fast64.go:92 +0x0
  main.generateNumbersMap.func1()
      /dev/webdevstation/blog-examples/concurent-map/main.go:68 +0xa4

Goroutine 8 (running) created at:
  main.generateNumbersMap()
      /dev/webdevstation/blog-examples/concurent-map/main.go:66 +0xb5
  main.main()
      /dev/webdevstation/blog-examples/concurent-map/main.go:33 +0x2f

Goroutine 7 (finished) created at:
  main.generateNumbersMap()
      /dev/webdevstation/blog-examples/concurent-map/main.go:66 +0xb5
  main.main()
      /dev/webdevstation/blog-examples/concurent-map/main.go:33 +0x2f
==================
==================
WARNING: DATA RACE
Read at 0x00c000146438 by goroutine 41:
  main.generateNumbersMap.func2()
      /dev/webdevstation/blog-examples/concurent-map/main.go:75 +0xc7

Previous write at 0x00c000146438 by goroutine 7:
  main.generateNumbersMap.func1()
      /dev/webdevstation/blog-examples/concurent-map/main.go:68 +0xb9

Goroutine 41 (running) created at:
  main.generateNumbersMap()
      /dev/webdevstation/blog-examples/concurent-map/main.go:73 +0x110
  main.main()
      /dev/webdevstation/blog-examples/concurent-map/main.go:33 +0x2f
==================

Found 2 data race(s)
exit status 66

There are several strategies that could be used to solve it. I will show one of them. We are going to introduce a new struct that provides it’s own mutex:

type SafeNumbers struct {
	sync.RWMutex
	numbers map[int]int
}

To be able to read and write items concurrently to this structure, we need to create the responsible methods:

func (sn *SafeNumbers) Add(num int) {
	sn.Lock()
	defer sn.Unlock()
	sn.numbers[num] = num
}

Here we are basically telling to lock the numbers map, during adding of the new number to it. Other goroutines will wait until it became unlocked again.

And another method for reading:

func (sn *SafeNumbers) Get(num int) (int, error) {
	sn.RLock()
	defer sn.RUnlock()
	if number, ok := sn.numbers[num]; ok {
		return number, nil
	}
	return 0, errors.New("Number does not exists")
}

Next, let’s refactor our generateNumbersMap() function:

func generateNumbersMap() {
	wg := sync.WaitGroup{}
    // Init our "safe" numbers map struct.
	safeNumbers := &SafeNumbers{
		numbers: map[int]int{},
	}
    // Write.
	for i := 0; i < 100; i++ {
		wg.Add(1)
		go func(i int) {
			defer wg.Done()
			safeNumbers.Add(i)
		}(i)
	}
    // Read.
	for i := 0; i < 100; i++ {
		wg.Add(1)
		go func(i int) {
			defer wg.Done()
			number, err := safeNumbers.Get(i)
			if err != nil {
				log.Print(err)
			} else {
				log.Print(number)
			}
		}(i)
	}

	wg.Wait()
}

If we run go run -race main.go now, there will be no more data race issues!

As I mentioned before, there also other ways to solve it. One of them is using of a special go type sync.Map.

Nevertheless, I hope this was helpful and you know now how to work safely with the maps in go. Especially, you should be careful with them when you create the web services, because every http request initiating a new goroutine.

As usual, the source code you can find here.

Related posts

Ristretto - the Most Performant Concurrent Cache Library for Go
2 Mar 2021

Ristretto - the Most Performant Concurrent Cache Library for Go

Recently, I discovered a surprisingly reliable memory caching solution, which I’m planning to use in all my further applications to increase performance. In this blog post, I will share some code examples of how you can integrate Ristretto caching library into your application.

go caching ristretto performance
How to Show Flash Messages in Go web applications (with Echo framework)
4 Feb 2021

How to Show Flash Messages in Go web applications (with Echo framework)

When we create a web application, usually, there a need to communicate with the users to inform them about the results of their actions. The easiest way to communicate - is to send messages. These messages might be warnings, errors, or just informational text. In this article, we will improve the UX of our user authentication application from the previous article by adding an error flash message when the user entered a wrong password and a success message after user authorisation.

go echo flash messages
User Authentication in Go Echo with JWT
28 Jan 2021

User Authentication in Go Echo with JWT

In this article, we will build a simple user authentication functionality using JWT (JSON Web Token). In the examples, I’m going to use a Go Echo framework. This will allow us to avoid writing some boilerplate code.

go authentication JWT Echo
A Simple Queue Implementation in Golang with channels
12 Jan 2021

A Simple Queue Implementation in Golang with channels

In this post, I’m going to show the way how we can implement a simple queue in Golang, using channels.

go queue channel goroutine
How to Sort Strings With Go Alphabetically in Any Language
4 Jan 2021

How to Sort Strings With Go Alphabetically in Any Language

In this article I’m going to show how easy we can sort strings alphabetically in different languages, using Go. It seems like an easy task if we want to sort English words, however, it’s not so trivial if we want to sort correctly strings with special characters or in other languages, i.e Cyrillic based.

go sorting alphabetical sort