Nim language

UPDATE: this review is outdated, there are ways to address almost all Nim problems I mentioned here, I'm going to write updated review in a couple of weeks.


I'm working on some data processing project that's currently using TypeScript and Kotlin. I had one free week and decided to explore if Nim could be a better choice.

So this comparison will be Nim vs Kotlin and TypeScript. I shall say that while Nim has better design and is a more powerful language, it also lacks proper support for some basic features. In my opinion, at the current moment - staying with Kotlin or TypeScript will be more productive. The project will be finished faster and there will be less errors than with Nim. Below I highlight some use cases and explain why I think so.

I also shall say that in this project simple and correct code is more important than the performance. That may be unusual requirement for Nim as it's more focussed on the performance.

Nim good features

  • Readable, clean and expressive code.
  • Multiple dispatch, uniform function call, named arguments.
  • Flexible data structures, objects, variants, tuples, sequences.
  • Everything is an expression.
  • First class functions.
  • Flexible type system with generics.
  • Good type inference, also helps to keep the code nice and clean.
  • Runs everywhere. C and JS backends cover most platforms, from Servers to Browsers.
  • High performance and small footprint.
  • Case and underscore insensitive, very nice feature.
  • Clean way to enforce immutability for variables, data structures and arguments.
  • Simple and well designed module system.
  • Simple package manager.
  • Flexible project file structure.
  • Simple and well designed way to manage code, project, executables etc.

Now I'm going to continue with the small example and problems I encountered.

The rest of the page could be summarized as - Nim should stop inventing advanced stuff and focus on improving the basics. While it has huge potential in its current state it's not easy to use.

Not clean output

Let's write simple program

echo "Hi"

The output

nim c -r test.nim
Hint: used config file '/usr/local/Cellar/nim/1.2.6/nim/config/nim.cfg' [Conf]
Hint: system [Processing]
Hint: widestrs [Processing]
Hint: io [Processing]
Hint: test [Processing]
CC: stdlib_io.nim
CC: test.nim
Hint:  [Link]
Hint: 14204 LOC; 0.422 sec; 20.258MiB peakmem; Debug build; proj: /alex/projects/nim/app.nim; out: /alex/projects/nim/test [SuccessX]
Hint: /alex/projects/nim/test  [Exec]
Hi

Lots of noise, not good, thankfully it seems like it's possible to silence it

nim c -r --hints:off test.nim
Hi

That's better, but to use libraries you need a package manager nimble, let's try it:

nimble run
  Verifying dependencies for nim@1
   Building nim/app using c backend
Hi

noise again, it will be much worse if you use libraries, like

$ nimble run app
  Verifying dependencies for nim@1
      Info: Dependency on serialization@any version already satisfied
  Verifying dependencies for serialization@0.1.0
      Info: Dependency on faststreams@any version already satisfied
  Verifying dependencies for faststreams@0.2.0
      Info: Dependency on stew@any version already satisfied
  Verifying dependencies for stew@0.1.0
      Info: Dependency on testutils@any version already satisfied
  Verifying dependencies for testutils@0.3.0
      Info: Dependency on chronos@any version already satisfied
  Verifying dependencies for chronos@2.5.1
      Info: Dependency on stew@any version already satisfied
  Verifying dependencies for stew@0.1.0
      Info: Dependency on bearssl@any version already satisfied
  Verifying dependencies for bearssl@0.1.5
      Info: Dependency on stew@any version already satisfied
  Verifying dependencies for stew@0.1.0
      Info: Dependency on json_serialization@any version already satisfied
  Verifying dependencies for json_serialization@0.1.0
      Info: Dependency on serialization@any version already satisfied
  Verifying dependencies for serialization@0.1.0
      Info: Dependency on faststreams@any version already satisfied
  Verifying dependencies for faststreams@0.2.0
      Info: Dependency on stew@any version already satisfied
  Verifying dependencies for stew@0.1.0
      Info: Dependency on testutils@any version already satisfied
  Verifying dependencies for testutils@0.3.0
      Info: Dependency on chronos@any version already satisfied
  Verifying dependencies for chronos@2.5.1
      Info: Dependency on stew@any version already satisfied
  Verifying dependencies for stew@0.1.0
      Info: Dependency on bearssl@any version already satisfied
  Verifying dependencies for bearssl@0.1.5
      Info: Dependency on stew@any version already satisfied
  Verifying dependencies for stew@0.1.0
      Info: Dependency on stew@any version already satisfied
  Verifying dependencies for stew@0.1.0
   Building nim/app using c backend
Hi

Not good, and this time there's no way to hide it and have nice and clean output Hi. Options like --hints:off --verbosity:0 didn't work.

That's a worrying sign, other people using Nim also seeing this output and they don't see that noise as problem, so it won't be fixed and improved in the future.

Object vs Ref Object

Nim has distinction between object and ref object and it creates extra complexities that don't exist in Kotlin or TypeScript.

It's nice to have low-level control to optimise critical code paths. But it's not good to being forced to deal with low-level details in the usual code.

Kotlin has high perfromance and achieves it without exposing low-level complexities.

Explicit ref memory allocation

More problem, if ref is used, the memory should be explicitly allocated. Bigger chance to make mistake. Compare this Nim code

raise new_exception(CatchableError, "Some error")

vs Kotlin

raise Exception("Some error")

Note that now you need to remember that some objects are not just plain objects but a special objects that have to be called in a speciall way.

Small example

I read Nim Tutorial and decided to convert some TypeScript code from my TypeScript and Kotlin codebase to Nim. And started with simple code for custom documentation

interface TextDoc {
  tags?:  string[]
  title:  string
  text:   string
}
interface TodoDoc {
  priority?: 'low' | 'normal' | 'high'
  tags?:     string[]
  todo:      string
}
type Doc = TextDoc | TodoDoc
const all_docs: Doc[] = []
function doc(doc: (Doc | (() => Doc))) {
  all_docs.push(typeof doc === 'function' ? doc() : doc)
}

I'm using it to write custom notes over code base like this one

doc({
  tags:  ['Math', 'Approximation'],
  title: 'Inverse Distance Weighting',
  text:  `
    Inverse Distance Weighted is a deterministic spatial interpolation ...
    ...
  `
})

and it would generate the following documentation

Let's convert it to Nim.

Using top-level source folder

The project I work with structured in lots of different directories, so I need to change the default structure for nimble project with single /src directory. The following changes need to be done in nim.nimble file

bin_dir     = "bin"
# src_dir    = "src"

Literal type can't be used

Let's declare our data structure for type Doc = TextDoc | TodoDoc in Nim.

type
  TextDoc = object
    title: string
    text:  string
    tags:  seq[string]

  TodoPriority = enum low, normal, high

  TodoDoc = object
    todo:     string
    priority: TodoPriority
    tags:     seq[string]

  Doc = TextDoc | TodoDoc

var all_docs: seq[Doc] = @[] # <= Error

It fails because Doc type can't be used in seq[Doc] the reference type seq[ref Doc] also can't be used with literal type.

UPDATE: In Nim it's not a literal type but typeclass and seems like it can't be used dynamically with seq.

Variant object type is not safe to use

Let's use Nim's object variant type

type
  DocKind = enum text, todo

  TodoPriority = enum low, normal, high

  Doc = object
    case kind: DocKind
    of text:
      title: string
      text:  string
    of todo:
      todo:     string
      priority: TodoPriority
    tags:  seq[string]

var all_docs: seq[Doc] = @[]

Works, but object variant type at least for now doesn't have good compile-type inspection.

The wrong code below were will be compiled and cause runtime exception:

echo Doc(kind: text, text: "some doc", tags: @["help"]).todo

The compiler didn't noticed that accessing the .todo property on kind: text type is invalid.

So we can't use object variant type as it doesn't have type safety. It's not a problem of a language though, the compiler will be improved in the future.

Forced to use OOP

So I can't use literal types or object variant and so Nim forces me to use OOP and hieararchy, not good. I would like to avoid OOP unless I really need it.

type
  Doc = object of RootObj

  TextDoc = object of Doc
    title: string
    text:  string
    tags:  seq[string]

  TodoPriority = enum low, normal, high

  TodoDoc = object of Doc
    todo:     string
    priority: TodoPriority
    tags:     seq[string]

var all_docs: seq[Doc] = @[]

all_docs.add TextDoc(title: "Some title", text: "Some text")

Error again, what's worse it's runtime eror, compiler failed to detect error at compile time. Such code is very common and that means we would have lots of runtime errors.

The problem is in this line, as TextDoc has different memory size and can't be stored in sequence of type seq[Doc]

all_docs.add TextDoc(title: "Some title", text: "Some text")

The reference should be used instead

No auto ref

Let's use ref

var all_docs: seq[ref Doc] = @[]

all_docs.add TextDoc(title: "Some title", text: "Some text")

Error again, all_docs.add fails to automatically convert TextDoc to ref TextDoc and I had to do that manually by writing helper function to_ref

proc to_ref[T](o: T): ref T =
  result.new
  result[] = o

var all_docs: seq[ref Doc] = @[]
all_docs.add TextDoc(title: "Some title", text: "Some text").to_ref

Not good, Nim forces me to think about low-level details of memory management, when in this case I don't care about performance as it's not a performance critical path. And even if it was I don't like to optimise code prematurely, before I see the numbers from the profiler.

Kotlin has almost same performance as Nim and manages to achieve that without ref complications.

Harder inspection

Let's try to inspect all_docs and add this line

echo all_docs.repr

There are two problems, first - simple echo all_docs won't work without repr and the second - the output is not clean, it's polluted with the refs noise we don't care about.

0x105a39048@[ref 0x105a38048 --> [todo = 0x105a38088"Something",
priority = normal,
tags = []]]

Conversion to JSON is not well supported

Now we need to save list of docs as JSON so it will be picked up by external tool and rendered into documentation, let's do that.

import json

type
  DocItem = object of RootObj
    tags: seq[string]

  TextDoc = object of DocItem
    text: string

  TodoDoc = object of DocItem
    todo: string

var docs: seq[ref DocItem]

proc to_ref[T](o: T): ref T =
  result.new
  result[] = o

docs.add TextDoc(text: "some doc", tags: @["help"]).to_ref
docs.add TodoDoc(todo: "some todo").to_ref

echo %docs

The ouptut is wrong, it fails to convert properties of child object correctly

[{"tags":["help"]},{"tags":[]}]

It should be

[{"text": "some doc", "tags": ["help"]}, {"todo": "some todo", "tags": []}]

It's possible to rewrite code as object variant and it will be converted to JSON properly. But as I mentioned earlier - in the current Nim version object variant are not type-safe.

import json

type
  DocKind = enum
    TextDoc, TodoDoc
  DocItem = object
    case kind: DocKind
    of TextDoc:
      text: string
    of TodoDoc:
      todo: string
    tags: seq[string]


var docs: seq[DocItem]

docs.add DocItem(kind:TextDoc, text: "some doc", tags: @["help"])
docs.add DocItem(kind:TodoDoc,todo: "some todo")

echo %docs

No autocast

At this stage, when it failed to do such a simple thing as to convert list of objects to json I gave up on Nim, but still decided to experiment a little bit more.

Let's try to print all the text docs from our list

for doc in docs:
  if doc of TextDoc: echo doc.text

It's not working because Nim doesn't support autocast and requires unsafe explicit cast (Kotlin and TS support autocast)

for doc in docs:
  if doc of TextDoc: echo cast[TextDoc](doc).text

No null-safety

The code below would cause runtime error (Kotlin and TS would detect null at compile time)

type
  RefDoc = ref object
    text: string

var v: RefDoc
echo v[]

Concept is in experimental state

Nim doesn't have interfaces but something called concept it seems to be more powerful way to define expected behavior than the interface but it still in experimental state.

Seems like there are couple of cases in my project when I may need to use it and that it's in the experimental state feels a bit worrying.

Inconsistent function call notation

Nim has very nice feature of optional braces in function calls and named arguments, so we can write

doc "Some", "Comment"
doc(title = "Some", text = "Comment")

Unfortunatelly, it's not working without braces with named argument, and this code would fail

doc title = "Some", text = "Comment"

Noisy error messages

In real project we going to use libraries for example

import serialization, json_serialization

And the real error we'll see will be not one-line but a huge mess where almost all of it is noise and only the single last line contains usefull infromation about the error, not good.

What's much worse - it seems, as I suppose everyone who uses Nim sees the same noisy error messages and don't complain - people don't see that as a problem and it won't be fixed.

$ nimble run test
  Verifying dependencies for nim@1
      Info: Dependency on serialization@any version already satisfied
  Verifying dependencies for serialization@0.1.0
      Info: Dependency on faststreams@any version already satisfied
  Verifying dependencies for faststreams@0.2.0
      Info: Dependency on stew@any version already satisfied
  Verifying dependencies for stew@0.1.0
      Info: Dependency on testutils@any version already satisfied
  Verifying dependencies for testutils@0.3.0
      Info: Dependency on chronos@any version already satisfied
  Verifying dependencies for chronos@2.5.1
      Info: Dependency on stew@any version already satisfied
  Verifying dependencies for stew@0.1.0
      Info: Dependency on bearssl@any version already satisfied
  Verifying dependencies for bearssl@0.1.5
      Info: Dependency on stew@any version already satisfied
  Verifying dependencies for stew@0.1.0
      Info: Dependency on json_serialization@any version already satisfied
  Verifying dependencies for json_serialization@0.1.0
      Info: Dependency on serialization@any version already satisfied
  Verifying dependencies for serialization@0.1.0
      Info: Dependency on faststreams@any version already satisfied
  Verifying dependencies for faststreams@0.2.0
      Info: Dependency on stew@any version already satisfied
  Verifying dependencies for stew@0.1.0
      Info: Dependency on testutils@any version already satisfied
  Verifying dependencies for testutils@0.3.0
      Info: Dependency on chronos@any version already satisfied
  Verifying dependencies for chronos@2.5.1
      Info: Dependency on stew@any version already satisfied
  Verifying dependencies for stew@0.1.0
      Info: Dependency on bearssl@any version already satisfied
  Verifying dependencies for bearssl@0.1.5
      Info: Dependency on stew@any version already satisfied
  Verifying dependencies for stew@0.1.0
      Info: Dependency on stew@any version already satisfied
  Verifying dependencies for stew@0.1.0
   Building nim/test using c backend
       Tip: 37 messages have been suppressed, use --verbose to show them.
     Error: Build failed for package: nim
        ... Details:
        ... Execution failed with exit code 1
        ... Command: "/usr/local/Cellar/nim/1.2.6/nim/bin/nim" c --noNimblePath -d:NimblePkgVersion=1 --path:"/Users/alex/.nimble/pkgs/serialization-0.1.0"  --path:"/Users/alex/.nimble/pkgs/faststreams-0.2.0"  --path:"/Users/alex/.nimble/pkgs/stew-0.1.0"  --path:"/Users/alex/.nimble/pkgs/testutils-0.3.0"  --path:"/Users/alex/.nimble/pkgs/chronos-2.5.1"  --path:"/Users/alex/.nimble/pkgs/stew-0.1.0"  --path:"/Users/alex/.nimble/pkgs/bearssl-0.1.5"  --path:"/Users/alex/.nimble/pkgs/stew-0.1.0"  --path:"/Users/alex/.nimble/pkgs/json_serialization-0.1.0"  --path:"/Users/alex/.nimble/pkgs/serialization-0.1.0"  --path:"/Users/alex/.nimble/pkgs/faststreams-0.2.0"  --path:"/Users/alex/.nimble/pkgs/stew-0.1.0"  --path:"/Users/alex/.nimble/pkgs/testutils-0.3.0"  --path:"/Users/alex/.nimble/pkgs/chronos-2.5.1"  --path:"/Users/alex/.nimble/pkgs/stew-0.1.0"  --path:"/Users/alex/.nimble/pkgs/bearssl-0.1.5"  --path:"/Users/alex/.nimble/pkgs/stew-0.1.0"  --path:"/Users/alex/.nimble/pkgs/stew-0.1.0"  -o:"/alex/projects/alien/old/nim/bin/test" "/alex/projects/alien/old/nim/test.nim"
        ... Output: Hint: used config file '/usr/local/Cellar/nim/1.2.6/nim/config/nim.cfg' [Conf]
        ... Hint: system [Processing]
        ... Hint: widestrs [Processing]
        ... Hint: io [Processing]
        ... Hint: test [Processing]
        ... Hint: serialization [Processing]
        ... Hint: typetraits [Processing]
        ... Hint: macros [Processing]
        ... Hint: macros [Processing]
        ... Hint: tables [Processing]
        ... Hint: hashes [Processing]
        ... Hint: math [Processing]
        ... Hint: bitops [Processing]
        ... Hint: algorithm [Processing]
        ... Hint: faststreams [Processing]
        ... Hint: inputs [Processing]
        ... Hint: os [Processing]
        ... Hint: strutils [Processing]
        ... Hint: parseutils [Processing]
        ... Hint: unicode [Processing]
        ... Hint: pathnorm [Processing]
        ... Hint: osseps [Processing]
        ... Hint: posix [Processing]
        ... Hint: times [Processing]
        ... Hint: options [Processing]
        ... Hint: memfiles [Processing]
        ... Hint: streams [Processing]
        ... Hint: ptrops [Processing]
        ... Hint: ptr_arith [Processing]
        ... Hint: async_backend [Processing]
        ... Hint: chronos [Processing]
        ... Hint: asyncloop [Processing]
        ... Hint: heapqueue [Processing]
        ... Hint: lists [Processing]
        ... Hint: nativesockets [Processing]
        ... Hint: net [Processing]
        ... Hint: sets [Processing]
        ... Hint: monotimes [Processing]
        ... Hint: ssl_certs [Processing]
        ... Hint: ospaths [Processing]
        ... Hint: deques [Processing]
        ... Hint: timer [Processing]
        ... Hint: selectors [Processing]
        ... Hint: kqueue [Processing]
        ... Hint: cstrutils [Processing]
        ... Hint: srcloc [Processing]
        ... Hint: asyncsync [Processing]
        ... Hint: handles [Processing]
        ... Hint: transport [Processing]
        ... Hint: datagram [Processing]
        ... Hint: common [Processing]
        ... Hint: stream [Processing]
        ... Hint: sendfile [Processing]
        ... Hint: ipnet [Processing]
        ... Hint: endians2 [Processing]
        ... Hint: osnet [Processing]
        ... Hint: asyncstream [Processing]
        ... Hint: chunkstream [Processing]
        ... Hint: debugutils [Processing]
        ... Hint: buffers [Processing]
        ... Hint: outputs [Processing]
        ... Hint: strings [Processing]
        ... Hint: pipelines [Processing]
        ... Hint: multisync [Processing]
        ... Hint: object_serialization [Processing]
        ... Hint: objects [Processing]
        ... Hint: errors [Processing]
        ... Hint: json_serialization [Processing]
        ... Hint: reader [Processing]
        ... Hint: strformat [Processing]
        ... Hint: types [Processing]
        ... Hint: lexer [Processing]
        ... Hint: json [Processing]
        ... Hint: lexbase [Processing]
        ... Hint: parsejson [Processing]
        ... Hint: writer [Processing]
        ... Hint: textio [Processing]
        ... /alex/projects/nim/src/app.nim(33, 19) Error: invalid indentation

Learning is not easy

I remember feeling when I switched from Java to Ruby. Many things in Ruby were new to me and alien to my old Java programming style. Yet, after couple of days I felt like my productivity going up and everything in Ruby works exactly as I expect and make sense.

Similar experience with TypeScript and Kotlin - I was able to learn it in a couple of days.

Nim - feels like hitting the wall. I thought I will convert this short JS docs code in half an hour - but get stuck for couple of days with surprises and unexpected behaviour on every step.

Conclusion

At this stage I decided to stop, as it seems there are problems:

1) Nim has weaker compile time inspection than both Kotlin and TypeScript. And the code is lower-level. That means there could be lots of runtime errors.

2) Nim has different preferences than me. I don't like such noisy error messages. It's a really huge red flag that language designers have different preferences. Maybe things I consider as noise has some important meaning for them. That means the language, the tool is not a good fit for my use case. So while Nim could be a good choice for someone it seems it's not a good fit for me.

3) Nim community don't see those issues as huge blockers, preventing people from using Nim, slowing its adoption, inflow of new users and contributors.

Notes

Those are my personal notes, please ignore it

  • Remember to check for the Killer App.