Counters and Async Infrastructures Proposal for Code Contribution - - PowerPoint PPT Presentation

Counters and Async Infrastructures

Proposal for Code Contribution

Guy Sela, Senior Engineer, HPE

Who am I ?

• Senior Engineer at HPE
• Served as a developer and team leader at the Israeli NSA
• Developing software since 2004

E-mail: guy.sela@hpe.com Linkedin: http://www.linkedin.com/pub/guy-sela/70/17/9aa

Agenda

Counters Infrastructure

• Why do we need Counters?
• Proposed Implementation
• Capabilities
• Examples
• Under the Hood
• Third-Party Alternatives

Async Infrastructure

• Scope
• Overview
• Under the Hood

Why do we need Counters?

• High-Frequency events will swamp the Log.
• Good alternative for Log.debug() and Log.trace()
• Useful information for troubleshooting.
• Powerful tool for analyzing bottlenecks.
• What events happened in my system until now?
• Analytics

Proposed Implementation

• Light-weight infrastructure that is very easy to integrate

into your code

• Supports the common use-cases for counters
• Found to be extremely useful for monitoring and

troubleshooting production systems

Capabilities

• Incremental & State Counters
• Configurable interval for “print counters delta to the log”
• Runtime counters silencing based on the user’s log library
• Almost no overhead on the system
• Counter querying API based on regular expressions
• REST API

Use Case – Incremental Counter

public class UserService { // High-Scale Event void userEvent() { log.info(“Event Called”); // Will swamp the Log … // Business Logic } }

Use Case – Incremental Counter

public class UserService { // High-Scale Event void userEvent() { UserServiceCounters.user_event_called.inc(); … // Business Logic } }

Log Output Example – 1 Minute Interval

[2016-02-16 14:18:27,626] INFO : (com.hpe.test.TestClassA:252) – Did Something [2016-02-16 14:18:27,967] INFO : (com.hpe.test.TestClassB:254) – Did Something Else [2016-02-16 14:18:28,028] CNT : user_event_called : +37824 [2016-02-16 14:18:45,125] INFO : (com.hpe.test.TestClassC:63) – Yet Another Log Line [2016-02-16 14:19:28,028] CNT : user_event_called : +41215 [2016-02-16 14:19:50,026] INFO : (com.hpe.test.TestClassC:72) – I Love Log Lines [2016-02-16 14:20:30,028] CNT : user_event_called : +51621 [2016-02-16 14:20:50,026] INFO : (com.hpe.test.TestClassA:11) – Log Lines Rock!

10

Log Output Example – Multiple Counters

[2016-02-16 15:18:27,626] INFO : (com.hpe.test.TestClassA:252) – Did Something [2016-02-16 15:18:27,967] INFO : (com.hpe.test.TestClassB:254) – Did Something Else [2016-02-16 15:18:28,028] CNT : user_event_called : +37824, other_counter: +72 [2016-02-16 15:18:45,125] INFO : (com.hpe.test.TestClassC:63) – Yet Another Log Line [2016-02-16 15:19:28,028] CNT : user_event_called : +41215, foo_counter: +17

11

public class UserService { // Boilerplate Code to define a new Counters Group enum UserServiceCounters { user_event_called(), other_user_counter(), third_counter(); private OccurenceCounter counter; public void inc() { counter.inc(); } UserServiceCounters() { counter = new OccurenceCounter(“UserServiceCounters”, name()); } } }

12

import com.hpe.counter.OccurrenceCounter;

Counter Types

• Incremental Counter – Value is manipulated using inc(). The

log prints show the delta from the last log print. No delta  No log print.

• State Counter – Value is manipulated using set(). The log

prints show the current value of the counter in case it

• changed. Useful for detecting queue build-up.

13

Counters Silencing

Log4J Config File // Standard mechanism to change Log Severity log4j.logger.UserService.user_event_called=WARN

14

Use Case - Using Counters to Identify Bottlenecks

void myBusinessLogic { String value = DB.getKey(“key”); // … Socket.sendOnSocket(new Packet(value.toBytes())); // … invokeIntensiveMethod(value); } Scenario: 5000 Events per second which invoke myBusinessLogic(); Symptom: We only manage to handle 2000 Events per second

15

Use Case - Using Counters to Identify Bottlenecks

void myBusinessLogic { BLCounters.before_db.inc(); String value = DB.getKey(“key”); BLCounters.after_db.inc(); // … BLCounters.before_socket.inc(); Socket.sendOnSocket(new Packet(value.toBytes())); BLCounters.after_socket.inc(); // … BLCounters.before_intensive.inc(); invokeIntensiveMethod(value); BLCounters.after_intensive.inc(); }

16

Log Output will tell us where is the Bottleneck

[2016-02-16 15:18:28,028] CNT : before_db: +5000, after_db: +2000, before_socket: +2000, after_socket: +2000, before_intensive: +2000, after_intensive: +2000

The DB is Slowing Us!

17

Log Output will tell us where is the Bottleneck

[2016-02-16 15:18:28,028] CNT : before_db: +5000, after_db: +5000, before_socket: +5000, after_socket: +5000, before_intensive: +5000, after_intensive: +2000

The Intensive Method is Slowing Us!

18

What events happened in my System until Now?

CountersDumperService API String getCounters(String regexp);

• Call using REST API / Other Management Console

19

Analytics

20

Under the Hood

• Counters based on AtomicLong & AtomicInteger causes

insignificant overhead.

• Using enum semantics instead of “register new counter”

implementation

• Background Thread that writes the delta of the counters to

the Log in each configurable interval.

21

Third-Party Alternatives

• Known third-party implementations with similar goals:

Metrics, Parfait, JAMon, Java Simon, Perf4J.

• We haven’t made a deep comparison between the

proposed solution to these.

22

Questions?

23

And Now to a Completely Different Subject

24

Async Infrastructure

25

Introduction

• In ODL’s Hydrogen version, every OSGI Bundle was

activated using the BundleActivator pattern.

• Every OSGI Service declared its implementations and

dependencies in its Activator.

• After the transformation to MD-SAL, Plugins/Applications

define their dependencies, implementations and API using YANG Model.

• Applications that reside in the Controller communicate with

each other using MD-SAL: RPCs and Notifications.

26

Introduction

• Application can be implemented by a number of OSGI Services

and Bundles.

• The presented Async Infrastructure facilitates communication

between OSGI services, and is agnostic to YANG, RPCs, etc.

• The infrastructure is currently only relevant for Bundles that still

use the BundleActivator pattern. Specifically for Activators that extend ComponentActivatorAbstractBase.

• If the presented capabilities interest the ODL community, we

can discuss extension of the infrastructure to support the new OSGI Bundle Activation patterns, that came along with MD-SAL

27

Scope

28

Async Infra (Unwanted?)

Application A

OSGI Service OSGI Service

MD-SAL

Async Infra

Application B

OSGI Service OSGI Service Async Infra

RPC / Notifications RPC / Notifications

Overview

29

• ServiceA implements PacketProcessingListener

Service A Service B Service C

doSomething() doSomethingElse()

• nPacketReceived() Notification

MD-SAL

Application A

public class ServiceA implements IServiceA, PacketProcessingListener { private IServiceB serviceB; public void onPacketReceived(PacketReceived notification) { serviceB.doSomething(); } }

30

public class ServiceB implements IServiceB { private IServiceC serviceC; public void doSomething() { serviceC.doSomethingElse(); } }

31

32

Service A Service B Service C

doSomething() doSomethingElse()

• The presented infrastructure enables you to convert these Java calls to Asynchronous

Multi-Threaded calls just by changing a configuration file!

Example of the config-file

/pools/OurThreadPool/poolSize=8 /handlers/com.hpe.ServiceB/PoolName=OurThreadPool /handlers/com.hpe.ServiceB/WorkerCount=5 /handlers/com.hpe.ServiceC/PoolName=OurThreadPool /handlers/com.hpe.ServiceC/WorkerCount=6

• We create a Thread Pool with 8 Threads, and define ServiceB and ServiceC to use it.
• ServiceB will use up to 5 concurrent Threads while ServiceC will use up to 6 concurrent

Threads.

33

Glimpse Under the Hood

34

ServiceA

doSomething()

ServiceB Proxy ServiceB Proxy

ServiceB Dedicated Queue Worker Thread Worker Thread Worker Thread

ServiceB

doSomething()

Benefits

• The developers can focus in coding the business logic of the Service.
• The API remains exactly the same with the help of Proxies and Reflection.
• Dynamically decide which services will be Async and control Thread Pools.
• No need to write bug-prone Boilerplate Code for Async method invocation. This

includes BlockingQueues, Producer Threads, Consumer Threads, Executors, Thread Pools, synchronized blocks. Caveat: The code itself obviously must support multi-threaded execution, i.e. Concurrent data structures, etc.

• The Queue can be capped, and messages that exceed the cap will be dropped.

Important for prevention of memory leaks.

35

What if I want only part of the API to be ASYNC?

public class MyService implements IMyService { public void methodWeWantAsync() { } public void anotherMethodWeWantAsync() { } @SyncMethod public void methodWeWantSync() { } } @SyncMethod - Will tell the infrastructure to run this method using the Caller’s Thread

36

What about invoking a Service in a remote ODL?

public class ServiceA implements IServiceA, PacketProcessingListener { private IServiceB serviceB; public void onPacketReceived(PacketReceived notification) { AsyncUtil.setTarget(new RemoteInstanceAddress(getIP())); serviceB.doSomething(); } private String getIP() { if (…) { return “127.0.0.1”; } else { return “1.2.3.4”; } } }

37

Glimpse Under the Hood

38

ServiceA

doSomething()

ServiceB Proxy ServiceB Proxy

RemoteAsync Service

call(ip, service, msg)

RemoteAsync Service

AKKA Sender

tell(serializedMsg)

ODL 1

Glimpse Under the Hood

39

doSomething()

ServiceB Proxy ServiceB Proxy

ServiceB Dedicated Queue Worker Thread Worker Thread Worker Thread

ServiceB

doSomething()

AKKA Receiver ODL 2

How Bi-Directional Invocations Work?

public class ServiceA implements IServiceA, PacketProcessingListener, IServiceBRepliesHandler { private IServiceB serviceB; public void onPacketReceived(PacketReceived notification) { AsyncUtil.setTarget(new RemoteInstanceAddress(getIP())); serviceB.doSomething(); } public void replyFromServiceB(Reply reply) { // Do logic } }

40

public class ServiceB implements IServiceB { private IServiceC serviceC; private IServiceBRepliesHandler replyHandler; public void doSomething() { serviceC.doSomethingElse(); AsyncUtil.setTarget(AsyncUtil.getCaller()); replyHandler.replyFromServiceB(new Reply()); } }

41

Caveats of the Bi-Directional Invocations

• By keeping the code clean from Async Java utils, we lose their potential
• capabilities. If the API of ServiceB would have been:

Future doSomething(); We could use Future.get(long Timeout), or Future.isDone(), etc.

• Reply code must contain the AsyncUtil.setTarget(AsyncUtil.getCaller()), but

this may be resolved in the future using Marker Annotation on the Reply

• Interface. Currently, this call looks like “magic” for a developer that doesn’t

know the underlying infrastructure.

42

Under the Hood

• When a Service registers using the BundleActivator, it must extend our new Activator:

AsyncAwareActivator. Example of Registration:

public class Activator extends AsyncAwareActivator { protected void registerServices() { registerNonAsyncService(SubscriberRegistrarDirect.class.getSimpleName(), SubscriberRegistrarDirect.class, // impls(impl(ISubscriberRegistrar.class), impl(ISystemLifecycle.class)), // dep(IDatabaseService.class), // dep(IGdms.class), // dep(ISubscriberGlobalControllerCurator.class));

} }

43

Under the Hood

• The infrastructure registers a Proxy to the Service in the OSGI Registry instead of the

Service itself.

• It creates a dedicated Queue<MethodCallMessage> for the Service’s Async Method Calls.

public class MethodCallMessage { Method method; Object[] args; }

These fields are populated from the Proxy call of java.lang.reflect.InvocationHandler: Object invoke(Object proxy, Method method, Object[] args)

44

Under the Hood

• It also create dedicated WorkerThreads and assign them

to a ThreadPool according to the configuration file shown previously.

• The WorkerThreads poll() the

Queue<MethodCallMessage> in a loop, and on each item polled they invoke: method.invoke(handlerInstance, args)

45

Under the Hood - Summary

• Using the illustrated capabilities, we harvest Java’s Proxy

and Reflection power, to allow a seamless conversion of normal method invocation to Async and potentially Remote invocation.

46

Follow-Up Discussions

• Adaptions of the infrastructure to fit the community needs.
• Absence of capabilities in the current MD-SAL Async

solution

• Allow customization of Thread Pools. Applications that want to

listen to Notifications on a specific ThreadPool. Applications that want its RPCs to run on a specific ThreadPool.

47

Questions?

48