Kamailio KEMI Framework – route logic migration to python

Abstract

In this article I will describe the usage of the KEMI framework on our Kamailio nodes. We’ve migrated all our http async requests to our API from the kamailio configuration scripting language to python. I’ve already described our dynamic dispatchers in Kamailio with jsonrpc and graphql with external Orchestrator and API. Please take a look at that article before proceeding.

Architecture overview

Our highly scalable cloud SIP infrastructure is using docker and kubernetes with microservices in kamailio, asterisk, rtpengine and cgrates. This kind of infrastructure can scale on premise, in the cloud with geographical regions request routing. It also has the possibility of having RTP nodes near the client installed on premise and it is surely a boost for the quality of experience and simplicity of deployment.

We have two distinct layers of kamailio nodes: the proxy and the router layer. Only the proxy layer has a public IP and is connected to the outside world on the Internet besides the media layer. Under the router layer we have a TPS (Transcoding and Playback Service) Asterisk layer and obviously external RTP nodes layer.

Dispatcher list reloading process

In the following schema you can observe the flow of the dynamic dispatcher reloading process that starts with a http rpc call from our orchestrator towards a kamailio node that triggers a graphql query on API to get a list of dispatchers.

This kind of approach is just a step towards a stateless kamailio instance that gets configured on runtime by the needs of the infrastructure as a whole. Creating multiple layers and handling the mutability of the whole by an orchestrator gives the ability to have a highly scalable and cloud oriented architecture.

Kamailio KEMI Framework

Kamailio uses a scripting language for it’s kamailio.cfg file which was developed from scratch and it’s a simil-C type of language with initial design going back to the year 2001.

The native scripting language meets it’s limitations with external services like our API.

The solution to meet our demands was found in the Kamailio Embedded Interface (KEMI) framework for executing SIP routing scripts in other programming languages.

This framework was added first in Kamailio v5.0.0 enabling multiple languages to be used and the interpreters for these languages are embedded in Kamailio, initialized at startup to be as fast as possible during runtime execution.

The kamailio.cfg still keeps the parts with:

  • Global parameters
  • Loading modules
  • Modules settings

These parts are evaluated once at startup and the majority of the parameters can be changed at runtime via RPC commands.

The languages supported by KEMI scripting are JavaScript, Lua, Python and Squrrel.

For more information about the KEMI Framework take a look at this article:

https://kamailio.org/docs/tutorials/devel/kamailio-kemi-framework/

Configuring Kamailio

It’s very simple to use python with KEMI, we only need two lines in our kamailio.cfg:

loadmodule "app_python.so"

modparam("app_python", "load", "/etc/kamailio/kamailio.py")

The modparam sets the initial python file where a class kamailio defines the functions that will be used in kamailio with the function python_exec. We will get to that later.

Writing kamailio.py

So our main python file imports all the libraries we need to use and returns the class kamailio with all it’s functions used with python_exec in the kamailio.cfg.

In the Python script we have to declare the global mod_init method where to instantiate an object of a class that implements the other callback methods (functions) to be executed by Kamailio.

import KSR as KSR

import json

import re


import gnr

import apiapp

import cgrates



def mod_init():

  return kamailio()



class kamailio:

  def child_init(self, rank):

    return 0


  def updateDispatchers(self, msg):

    try:

      token = getToken()



      dl = apiapp.dispatcherList(token)


      dispatcherList =

        getJsonPath("data.kamailio.dispatcher.list.kamailioConf", dl)


      if(dispatcherList):

        with open("/tmp/dispatcher.list", "w") as dlFile:

    dlFile.write(dispatcherList)

     except:

       kamExit("updateDispatchers failed!")

     return 1

# -- END class kamailio -- #

As you can see the the updateDispatchers function will be called inside our kamailio.cfg, but we will get to that later. In the same file we have defined also the getToken function that reads from the shared table the token otherwise it just loads a new one from the API and stores it to the shared table for future use.

The shared table for the token has an autoexpire of 50 minutes while the JavaScript Web Token expires an hour time.

modparam("htable", "htable", "api=>size=2;autoexpire=3000;")

So let’s take a look at the getToken function:

def getToken():

  """Checks the token availability in the shared table.

  If there is no token gets a new one from API and saves it to the sht.

  """

  token = KSR.pv.get("$sht(api=>token)")


  if(token is None):

    KSR.xlog.xlog("L_INFO", "Refreshing api token\n")

      try:

        newToken = apiapp.getApiToken()

         if(newToken["success"]):

           KSR.pv.sets("$sht(api=>token)", newToken["token"])

           token = newToken["token"]

         else:

           KSR.xlog.xerr("{}".format(json.dumps(newToken)))

      except:

        kamExit("getToken failed!")


  return token

Quite simple to understand, now let’s see the apiapp.py functions included in the kamailio.py:

import os

import json

import httplib


apiCredentials=""

with open("/tmp/apiCredentials", "r") as cred:

apiCredentials=cred.read().replace("\n", "")



dispatcherListQuery = """

{ kamailio { dispatcher { list { kamailioConf } } } }

"""


def getApiToken():

hdr = {"content-type": "application/x-www-form-urlencoded" }

conn = httplib.HTTPConnection("api-app-service:9016")

conn.request("POST", "/auth", apiCredentials, hdr)



response = conn.getresponse()

data = response.read()



return json.loads(data)



def graphqlQuery(apiToken, query):

payload = {}

payload["query"] = query



hdr = {

"content-type": "application/json",

"Authorization": "Bearer {token}".format(token=apiToken)

}


conn = httplib.HTTPConnection("api-app-service:9016")

conn.request("POST", "/graphql", json.dumps(payload), hdr)



response = conn.getresponse()

data = response.read()


return json.loads(data)


def dispatcherList(token):

return graphqlQuery(token, dispatcherListQuery)

With dispatcherList function we’re querying the graphql endpoint from the origin updateDispatchers function in the kamailio.py seen above.

Using python function in the xhttp route

At this point we’ve rewritten all the kamailio route logic seen in the previous article (DISPATCHER_LIST and DISPATCHER_SET) into a simpler python code with no need to use the HTTP_ASYNC_CLIENT module.

Our xhttp request route will be the following:

event_route[xhttp:request] {

  if!(dst_port==80) {

    xlog("L_NOTICE", "[XHTTP:REQUEST] $si FORBIDDEN! ***\n");

    exit;

  }

  if ($hu =~ "^/rpc") {

    xlog("L_NOTICE", "[XHTTP:REQUEST] $si ACCEPTED ***\n");

    jansson_get("method", "$rb", "$var(rpcMethod)");

    xlog("L_NOTICE", "[XHTTP:REQUEST] RPC METHOD: $var(rpcMethod) ***\n");

    if($var(rpcMethod) == "dispatcher.reload") {

      xlog("L_NOTICE", "Reloading dispatcher list\n");

      python_exec("updateDispatchers");

    }

  }

  jsonrpc_dispatch();

  exit;

}

The orchestrator http request towards our kamailio node is the following:

url: `http://${pod.ipAddress}/rpc`,

body: JSON.stringify({'jsonrpc': '2.0', 

                      'method': 'dispatcher.reload', 'id': '1'})

As you can see the jansson_get function loads the dispatcher.reload RPC function that is triggered by the jsonrpc_dispatch function after the python_exec(“updateDispatchers”) function fetches the new dispatcher list and writes it to our /tmp/dispatcher.list. The same file used by the dispatcher module:

modparam("dispatcher", "list_file", "/tmp/dispatcher.list")

Conclusions

This example shows you the possibility to use python (or any other KEMI supported language) for functions that are easier to read and develop by programmers knowing the basics of kamailio and obviously python.

For external services using modern technology solutions it’s surely simpler to develop in a high-level programming languages than using native kamailio scripting language. We’ve managed to use both but find it easier to just use python for some parts of the routing logic.

 

Leave a Reply

Your email address will not be published. Required fields are marked *