Using Neo4j graph database: part 2

In the ‘Using Neo4j graph database: part 1’ blog post by InternetDevels Drupal web developer, we have learned the basics of Neo4J. Now, let’s move on to more practical Neo4J use.

In this part, we will create an application that will return the data from Neo4j (the nearest airports according to the given coordinates) in the Json format. For this application, we will use the Phalcon framework. In our example, Neo4j contains the data on 8 thousand airports and a spatial index for these airports. For caching, we will use Redis.

Phalcon

Phalcon is one of the fastest PHP frameworks. All components are written in C, there is support for MVC, ORM, and Volt template engine is available (which resembles Jinja).

Installation instructions (for Ubuntu):

sudo apt-add-repository ppa:phalcon/stable

sudo apt-get update

sudo apt-get install php5-phalcon

find more details here: https://phalcon.io/en-us/download/linux

Then create the application itself.

Basic structure:

route-finder.loc

── app

│ ├── controllers

│ ├── models

│ └── views

└── public

       ├── css

       ├── img

       └── js

Create a .htaccess file in the application root folder and in the public folder. Next, create index.php in the public folder (do not forget to change the base URI). Create a app/controllers/IndexController.php controller, as in the example. If everything is done right, then, at this address localhost/route-finder.loc, you will see the following:

So now you have a page. Try to output some information from Neo4j to this page. To do this, install the neo4jphp library using Composer:

In the console (the application root) enter the following:

composer require "everyman/neo4jphp" "dev-master"

At this stage, the application structure should look like this:

route-finder.loc

├── app

│ ├── controllers

│ │ └── IndexController.php

│ ├── models

│ └── views

├── composer.json

├── composer.lock

├── composer.phar

├── public

│ ├── css

│ ├── img

│ ├── index.php

│ └── js

└── vendor

...

The IndexController.php file will look like this:

 <?php

use Phalcon\Mvc\Controller;

class IndexController extends Controller
{

    public function indexAction()
    {
        echo "<h1>Hello!</h1>";
    }
}

Change it in the following way:

 <?php
require_once __DIR__. '/../../vendor/autoload.php';

use Phalcon\Mvc\Controller;

class IndexController extends Controller
{

    public function indexAction()
    {
        $client = new Everyman\Neo4j\Client('localhost', 7474);

        $this->view->disable();

        //Create a response instance
        $response = new \Phalcon\Http\Response();

        //Set the content of the response
        $response->setContent(json_encode($client->getServerInfo()));
        $response->setContentType('application/json', 'UTF-8');

        //Return the response
        return $response;
    }
}

Explanation of code:

require_once __DIR__. '/../../vendor/autoload.php'; — enable autololad;

$client = new Everyman\Neo4j\Client('localhost', 7474); — connect to Neo4j. If you need to take the data from Neo4j in other routes as well, then it is better to inject Neo4j as a service (then the data will be available in all routes);

$this->view->disable(); — the views is not needed, since the controller itself will return the response;

$response->setContent(json_encode($client->getServerInfo())); — return something from Neo4j;

$response->setContentType('application/json', 'UTF-8'); — return as Json.

Now the main page will look like this:

You have a Neo4j base with airports and an html page that will provide your application with the longitude and latitude data for finding the 5 nearest airports. The query will be sent in this format: /ajax/airports?lat=50.433333&lng=30.516667&limit=5

Expected answer:

First, create a route. For this, add the following to the index.php file:

 $di->set('router', function () {

        $router = new \Phalcon\Mvc\Router();

        $router->add("/ajax/airports", array(
            'controller' => 'airports',
            'action' => 'index',
        ));

        return $router;
    });

The index.php will now look like this:

 <?php

use Phalcon\Loader;
use Phalcon\Mvc\View;
use Phalcon\Mvc\Application;
use Phalcon\DI\FactoryDefault;
use Phalcon\Mvc\Url as UrlProvider;
use Phalcon\Db\Adapter\Pdo\Mysql as DbAdapter;

try {

    // Register an autoloader
    $loader = new Loader();
    $loader->registerDirs(array(
        '../app/controllers/',
        '../app/models/'
    ))->register();

    // Create a DI
    $di = new FactoryDefault();

    // Setup the view component
    $di->set('view', function () {
        $view = new View();
        $view->setViewsDir('../app/views/');
        return $view;
    });

    // Setup a base URI so that all generated URIs include the "route-finder.loc" folder
    $di->set('url', function () {
        $url = new UrlProvider();
        $url->setBaseUri('/route-finder.loc/');
        return $url;
    });

    $di->set('router', function () {

        $router = new \Phalcon\Mvc\Router();

        $router->add("/ajax/airports", array(
'namespace' => 'RouteFinder',
            'controller' => 'airports',
            'action' => 'index',
        ));

        return $router;
    });

    // Handle the request
    $application = new Application($di);

    echo $application->handle()->getContent();

} catch (\Exception $e) {
     echo "PhalconException: ", $e->getMessage();
}

Next, create a copy of the IndexController.php file and name it AirportsController.php in the app/controllers folder. If you have done everything right, the /ajax/airports page will look similar to the main page.

Now, copy this code there:

<?php
require_once __DIR__. '/../../vendor/autoload.php';

use Phalcon\Mvc\Controller;
use Everyman\Neo4j\Client;
use Everyman\Neo4j\Cypher;

header('Access-Control-Allow-Origin: *');

function _distance($lat1, $lon1, $lat2, $lon2)
{
    $theta = $lon1 - $lon2;
    $dist = sin(deg2rad($lat1)) * sin(deg2rad($lat2)) +  cos(deg2rad($lat1)) * cos(deg2rad($lat2)) * cos(deg2rad($theta));
    $dist = acos($dist);
    $dist = rad2deg($dist);
    return $dist * 111.18957696;
}


class AirportsController extends Controller
{

    public function indexAction()
    {
        $client = new Client('localhost', 7474);

        $lat = number_format($this->request->getQuery("lat", "float", 0.0), 6);
        $lng = number_format($this->request->get('lng', "float", 0.0), 6);

        $result = array();


        $query = sprintf(
            "START n=node:geom('withinDistance:[%s, %s, %s]')
            MATCH n-[r:AVAILABLE_DESTINATION]->()
            RETURN DISTINCT n
            SKIP %s LIMIT %s",
            $lat,
            $lng,
            number_format($this->request->get('distance', "float", 500), 1),
            $this->request->get('offset', "int", 0),
            $this->request->get('limit', "int", 5)
        );

        $query = new Cypher\Query($client, $query);

        $query = $query->getResultSet();

        foreach ($query as $row) {
            $item = array(
                'id' => $row['n']->getId(),
                'distance' => _distance(
                    $row['n']->getProperties()['latitude'],
                    $row['n']->getProperties()['longitude'],
                    $lat,
                    $lng
                ),
            );
            foreach ($row['n']->getProperties() as $key => $value) {
                $item[$key] = $value;
            }
            $result[] = $item;
        }
        $response = new \Phalcon\Http\Response();

        //Set the content of the response
        $response->setContent(json_encode(array('json_list' => $result)));
        $response->setContentType('application/json', 'UTF-8');

        return $response;
    }
}

Explanation of code:

header('Access-Control-Allow-Origin: *'); — allow cross-domain queries;

function _distance($lat1, $lon1, $lat2, $lon2) — an auxiliary function to calculate the distance according to the given coordinates;

$lat = number_format($this->request->getQuery("lat", "float", 0.0), 6); — save the "lat" query parameter in the variable, in addition, format it as a number with 6 decimal places, because in Neo4j it is required that floating-point numbers should have at least one decimal place;

$query = sprintf( — here, set a template for the query to Neo4j, using the Cypher query language (you can read more about Neo4j and Cypher in the previous article). With this query, you can find the nodes that are closest to the given coordinates (for this, SpatialPlugin for Neo4j is installed and the corresponding index is created);

$query = new Cypher\Query($client, $query); — sending the query;

$query = $query->getResultSet(); — getting the results;

foreach ($query as $row) { — formatting the results;

'id' => $row['n']->getId(), — formatting the node ID using the getId() method;

'distance' => _distance( — finding the distance between the current airport and the given point using the above-mentioned auxiliary function;

foreach ($row['n']->getProperties() as $key => $value) { — adding every available property of the node to the array.

So we created an application that will return data about the nearest airports for the given coordinates, like this:

The answer in this case takes around 200ms.

Let's add caching in order to reduce the response time with the help of Redis.

1. Install and configure Redis, if it is not installed.

Instructions can be found here - https://www.digitalocean.com/community/tutorials/how-to-configure-a-redis-cluster-on-ubuntu-14-04

2. Install a php extension for Redis:

sudo pecl install redis

3. Add the following line to php.ini:

extension=redis.so

4. Restart your web server.

5. Change the index.php in the following way:

 <?php

use Phalcon\Loader;
use Phalcon\Mvc\View;
use Phalcon\Mvc\Application;
use Phalcon\DI\FactoryDefault;
use Phalcon\Mvc\Url as UrlProvider;
use Phalcon\Db\Adapter\Pdo\Mysql as DbAdapter;
use Phalcon\Cache\Backend;

try {

    // Register an autoloader
    $loader = new Loader();
    $loader->registerDirs(array(
        '../app/controllers/',
        '../app/models/'
    ))->register();

    // Create a DI
    $di = new FactoryDefault();

    // Setup the view component
    $di->set('view', function () {
        $view = new View();
        $view->setViewsDir('../app/views/');
        return $view;
    });

    // Setup a base URI so that all generated URIs include the "route-finder.loc" folder
    $di->set('url', function () {
        $url = new UrlProvider();
        $url->setBaseUri('/route-finder.loc/');
        return $url;
    });

    $di->set('router', function () {

        $router = new \Phalcon\Mvc\Router();

        $router->add("/ajax/airports", array(
            'controller' => 'airports',
            'action' => 'index',
        ));

        return $router;
    });

Explanation of code:

$di->set('cache', function () { — add the cache to the application

6. Change the AirportsController.php in the following way:

 <?php
require_once __DIR__. '/../../vendor/autoload.php';

use Phalcon\Mvc\Controller;
use Everyman\Neo4j\Client;
use Everyman\Neo4j\Cypher;

header('Access-Control-Allow-Origin: *');

function _distance($lat1, $lon1, $lat2, $lon2)
{
    $theta = $lon1 - $lon2;
    $dist = sin(deg2rad($lat1)) * sin(deg2rad($lat2)) +  cos(deg2rad($lat1)) * cos(deg2rad($lat2)) * cos(deg2rad($theta));
    $dist = acos($dist);
    $dist = rad2deg($dist);
    return $dist * 111.18957696;
}


class AirportsController extends Controller
{

    public function indexAction()
    {
        $client = new Client('localhost', 7474);

        $lat = $this->request->getQuery("lat", "float", 0.0);
        $lng = $this->request->get('lng', "float", 0.0);
        $distance = $this->request->get('distance', "float", 500);
        $offset = $this->request->get('offset', "int", 0);
        $limit = $this->request->get('limit', "int", 5);

        $cache =& $this->di->get('cache');

        $redis_key = implode('|', array(
            'AirportsController',
            'indexAction',
            $lat,
            $lng,
            $distance,
            $offset,
            $limit,
        ));

        try {
            // If Redis is connected - try to get result.
            $result = $cache->get($redis_key);
            $cache_connected = true;
        } catch (Exception $e) {
            $result = false;
            $cache_connected = false;
        }

        if ($result) {
            $result = unserialize($result);
        } else {
            $result = array();

            $query = sprintf(
                "START n=node:geom('withinDistance:[%s, %s, %s]')
                MATCH n-[r:AVAILABLE_DESTINATION]->()
                RETURN DISTINCT n
                SKIP %s LIMIT %s",
                number_format($lat, 6),
                number_format($lng, 6),
                number_format($distance, 1),
                $offset,
                $limit
            );

            $query = new Cypher\Query($client, $query);

            $query = $query->getResultSet();

            foreach ($query as $row) {
                $item = array(
                    'id' => $row['n']->getId(),
                    'distance' => _distance(
                        $row['n']->getProperties()['latitude'],
                        $row['n']->getProperties()['longitude'],
                        $lat,
                        $lng
                    ),
                );
                foreach ($row['n']->getProperties() as $key => $value) {
                    $item[$key] = $value;
                }
                $result[] = $item;
            }

            if ($cache_connected) {
                $cache->set($redis_key, serialize($result));
            }
        }

        $this->view->disable();
        $response = new \Phalcon\Http\Response();

        // Set the content of the response.
        $response->setContent(json_encode(array('json_list' => $result)));
        $response->setContentType('application/json', 'UTF-8');

        return $response;
    }
}

Explanation of code:

$redis_key = implode('|', array( — generating a unique key for Redis by combining the class name, the method name and the parameters;

try { — using “try” so that, in case of an unsuccessful attempt to connect to Redis, you get no errors and the service works without Redis;

$result = $cache->get($redis_key); — trying to get the value from Redis according to the previously generated key;

$cache_connected = true; — if the previous line didn’t call Exception, then Redis is available and you save “true” in the variable;

} catch (Exception $e) { — the code below will be run if there was an Exception;

$cache_connected = false; — Redis is unavailable and you save false in the variable;

$result = unserialize($result); — if you got the value from Redis, then you deserialize it and put in the variable;

$cache->set($redis_key, serialize($result)); — if Redis is enabled, you save the serialized value according to the previously generated key (the code is run only if the value according to the previously generated key was not in Redis).

Due to this, the response time for the uncached result will barely change, and the cached result will be returned in about 7ms. Thus, we reduced the response time by about 30 times and relieved Neo4j. In addition, the application will work correctly without Redis.

Here's the repository with the described application — https://github.com/mikekeda/route-finder

Hope these tips have been useful and good luck to you in using Neo4j graph database!