Universal Data Analytics as Semantic Spacetime(Part 2)


Part 2: Setting up the Tools

I confess to having a love and hate relationship with tech. In an ideal world, one could skip this post, because software would be easy to get started with, and the answers to simple questions would be easy to find — but that’s not always the case. As a researcher first and technologist second, my patience for geeking out over someone else’s tech is limited. I want to explore ideas by playing around on a laptop, free of the need to pay for cloud servers, or install container runtime environments with all the associated overhead. There are two things we need, in particular, to be able to play with Semantic Spacetime models: a programming language and a flexible data store. In this post I’ll show how to set up those.

Feel free to skip this episode if you already have ArangoDB and Go up and running. Also please note that code formatting in medium is rather stupid so the rendering might need some interpretation!

Programming language

Although I had imagined starting with popular Python to explain the concepts of semantic spacetime, I’ve instead chosen the ugly-mug language “Go” (or Golang), both for its speed and simplicity, and for its mature language driver. Having a stable and mature driver is a big issue in database usage, because otherwise databases are a pain to work with. Go could hardly be called pretty, but it has several advantages over Python, and its ad hoc and opinionated rules, which will infuriate from time to time, are tolerable. Go really has what you need, at your fingertips, so it’s a good investment for speed and efficiency.

Data store

Next we need a convenient data store, or some kind of database — one that doesn’t require a three year training certificate to operate. After playing with numerous implementations of Semantic Spacetime over the past decade, using MySQL, MongoDB, Postgres, Neo4j, and eventually plain filesystem implementations, a former research student stumbled across ArangoDB, which suddenly made difficult things easy. ArangoDB has therefore become my database of choice for a wide range of things. It’s a multi-model database system, which works as a key-value, document, and graph data store — for flexible storage, so we can choose what’s best without having to learn multiple technologies. It also has quite good language support. In fact, Arango seems poised to become much more than a database system — it’s being extended into a pretty powerful integrated data processing platform, with nascent support for user extensions, processing algorithms and “Pregel ‘’ graph flow computation models — executed across clusters. It feels like a good investment, even for someone of limited tolerance for tech features and details like me.

In what follows, I’ll assume you’re using Linux or a Mac with a shell environment. I don’t expect Windows to be a problem, but that’s not my world. Linux is by far the simplest environment in which to do research programming.

Installation

So we have two getting started challenges:

  • Installing Go(lang) on localhost

  • Installing ArangoDB on localhost

Both of these can be downloaded as easy-to-install packages. Just follow the “destructions”! We just start by searching the Internet for those packages (they don’t seem to be available through the Linux distributions). You can find the Go language here with steps to install:

https://golang.org/dl/

After installing a package for your operating system, you need to set up some things in your environment so that you can forget about golang for the rest of your tortured life. One less thing to fret over.

You’ll need a command window (shell). Then create some directories for the Golang workspace. These are used to simplify the importing of packages.

% mkdir -p ~/go/bin% mkdir -p ~/go/src% git clone https://github.com/markburgess/SemanticSpaceTime% ln -s ~/clonedirectory/pkg/SST ~/go/src/SST

The last step links the directory where you will keep the Smart Spacetime code library to the list of libraries that Go knows about. You’ll also need to set a GOPATH environment variable and add the installation directory to your execution path.For Linux (using default bash shell) you edit the file “~/.bashrc” in your home directory using your favourite text editor. It should contain these lines, as per the golang destructions:

export PATH=$PATH:/usr/local/go/binexport GOPATH=~/go# Set a short promptexport PS1=”mark%

Don’t forget to restart your shell or command window after editing this.

Since version 1.13 of Go, big changes have been made (and are expected to continue going forwards, sigh) concerning “modules” design. Unless you know what you’re doing, disable modules by running:

% go env -w GO111MODULE=off

To use the Go Driver, download it

% go get github.com/arangodb/go-driver

Testing Go



Continue Reading...