experiences building influxdb in go
play

Experiences Building InfluxDB in Go Paul Dix paul@influxdb.com - PowerPoint PPT Presentation

Experiences Building InfluxDB in Go Paul Dix paul@influxdb.com @pauldix Who am I? informs my experience working with a language like Go co-founder, CEO, programmer Author Languages worked with Professionally in order VBScript Delphi


  1. Experiences Building InfluxDB in Go Paul Dix paul@influxdb.com @pauldix

  2. Who am I? informs my experience working with a language like Go

  3. co-founder, CEO, programmer

  4. Author

  5. Languages worked with Professionally in order • VBScript • Delphi • C# and VB.NET • Ruby & Javascript • Java • C (only a little) • Ruby & Javascript • Scala • Go

  6. Mix of static and dynamic, but most time in dynamic languages

  7. Open source time series database written in Go

  8. What’s a time series?

  9. Stock trades and quotes

  10. Metrics

  11. Analytics

  12. Events

  13. Sensor data

  14. Two kinds of time series data…

  15. Regular time series t7 t6 t3 t4 t2 t1 t0 Samples at regular intervals

  16. Irregular time series t7 t6 t3 t4 t2 t1 t0 Events whenever they come in

  17. Why’d we pick Go?

  18. Some project requirements • Self contained binary install

  19. Some project requirements • Self contained binary install • Performance

  20. previous experience with it

  21. faster development than working with C/C++

  22. growing community

  23. simplicity of the language significant advantage for picking up new programmers and contributors

  24. InfluxDB Project Stats • First Commit - September 26th, 2013 • 176 Contributors • 68,000 LOC

  25. Team background • Java • Scala • Python • Ruby • C++

  26. What has been great

  27. Static typing can’t believe I’m saying this

  28. Community

  29. Performance

  30. What surprised us

  31. GC hasn’t been a problem

  32. Contributors with no previous Go experience simplicity of the language wins again

  33. Haven’t really missed generics

  34. except when I do…

  35. // Sort methods func (a Values) Len() int { return len(a) } func (a Values) Swap(i, j int) { a[i], a[j] = a[j], a[i] } func (a Values) Less(i, j int) bool { return a[i].Time().UnixNano() < a[j].Time().UnixNano() }

  36. Duplicate code for each data type select percentile(90, value ) from cpu where time > now() - 1d group by time (10m)

  37. // Iterator represents an iterator over a series. type Iterator interface { SeekTo(seek int64) (key int64, value interface {}) Next() (key int64, value interface {}) Ascending() bool } we have to cast this later

  38. Costs on performance f := val.(float64) // do some math

  39. // FloatIterator represents an iterator over a series. type FloatIterator interface { SeekTo(seek int64) (key int64, value float64) Next() (key int64, value float64) Ascending() bool }

  40. Implement for every function func MapMean(input *MapInput) interface {} { if len(input.Items) == 0 { return nil } out := &meanMapOutput{} for _, item := range input.Items { out.Count++ switch v := item.Value.( type ) { case float64: out.Total += v case int64: out.Total += float64(v) out.ResultType = Int64Type } } return out }

  41. Could use interfaces, but there’s a performance penalty there too…

  42. What has been bad

  43. Dependency management

  44. Experiences migrating to 1.5

  45. Compile times longer on a project this size it’s noticeable

  46. Performance more about our code

  47. Go 1.5 issue forced us to revert https://github.com/golang/go/issues/12233

  48. Unit and integration tests won’t save you. if you operate at scale, only full blown scale tests will tell you anything and even then you may not find it

  49. Q&A paul@influxdb.com @pauldix

Recommend


More recommend