• March 1, 2016
• David Marks
• 0

INF5100 Mandatory

Exercise

9/30/2015

Question 1:

What are some typical uses for complex event processing applications? Give examples and elaborate on the possibilities.

Some typical uses for complex event processing are air traffic control, stock market trading, business process management and fraud detection.

One of the CEP uses is to analyze data in real time i.e we get data continuously and we analyze it at the same time, so we dont have to store it first, and then analyze it, it

give us the results much faster than old classical database systems which is a great advantage, so we can use it for crucial system to check data continuously and give quick response if something goes wrong.

CEP can be used to track productions of big companies to presume their next year production based on the gathered, current data.

In stock market a buyer may place an order depending on many factors like average price over last hour, total turn over for the particular symbol, etc. All these are events which can be processed using a CEP suite like Esper.

It can also be used for making global policies based on events throughout the world.

Question 2:

Find the date of the BTC stock when closing price is greater than 0.1 and less than 20. (QUERY + RESULTS + GRAPH)

Query: SELECT timestamp, close FROM StockTick WHERE close > 0.1 AND close < 20

Result: EVENT! {close=0.105, timestamp=Wed Oct 13 00:00:00 IST 2010} EVENT! {close=0.102, timestamp=Thu Oct 14 00:00:00 IST 2010} EVENT! {close=0.105, timestamp=Fri Oct 15 00:00:00 IST 2010}

EVENT! {close=0.101, timestamp=Sat Oct 16 00:00:00 IST 2010} EVENT! {close=0.102, timestamp=Sun Oct 17 00:00:00 IST 2010} EVENT! {close=0.1024, timestamp=Mon Oct 18 00:00:00 IST 2010} EVENT! {close=0.107, timestamp=Thu Oct 21 00:00:00 IST 2010} EVENT! {close=0.1025, timestamp=Fri Oct 22 00:00:00 IST 2010} EVENT! {close=0.1055, timestamp=Sat Oct 23 00:00:00 IST 2010} EVENT! {close=0.11501, timestamp=Sun Oct 24 00:00:00 IST 2010}…

EVENT! {close=19.52538, timestamp=Tue Jan 29 00:00:00 IST 2013} EVENT! {close=19.70001, timestamp=Wed Jan 30 00:00:00 IST 2013} EVENT! {close=19.63001, timestamp=Sat Feb 02 00:00:00 IST 2013}

Graph:

Question 3:

What is the difference between doing aggregated queries (AVG/SUM/MAX/ETC) in standard databases and stream databases?

The difference between doing aggregated queries in standard database system and stream database system is that, standard database queries are one time queries or static queries, and stream database queries are continues queries or real time queries, in standard database system we have stored the data on a disk before performing queries on it, we have almost all information about the data before performing queries on it, so we can easily access the data and verify the result, we get a single result when we do aggregated queries in standard database system,

But stream database system is real time, the data comes and goes out of the system continuously, we dont store the data first and we dont have any information about

the data, we just make a general schema for it and process it as it comes into the system, so the result of the continuous queries on stream database system will be changing continuously according to the data that the system receives.

Question 4:

For each non-overlapping 100 tick interval of the BTC stock, find the minimum and maximum weighted price in the interval.

Thoughts:

Esper is very easy for analyzing a set of events and we can filter by time or length.

Query:

SELECT MIN(weightedPrice), MAX(weightedPrice) FROM

StockTick.win:length_batch(100)

Result:

EVENT! {max(weightedPrice)=0.36, min(weightedPrice)=0.06} EVENT! {max(weightedPrice)=1.06, min(weightedPrice)=0.19} EVENT! {max(weightedPrice)=8.67, min(weightedPrice)=0.67} EVENT! {max(weightedPrice)=29.58, min(weightedPrice)=5.03} EVENT! {max(weightedPrice)=6.11, min(weightedPrice)=2.14} EVENT! {max(weightedPrice)=7.05, min(weightedPrice)=3.79} EVENT! {max(weightedPrice)=6.71, min(weightedPrice)=4.72} EVENT! {max(weightedPrice)=13.26, min(weightedPrice)=6.77} EVENT! {max(weightedPrice)=17.15, min(weightedPrice)=10.31} EVENT! {max(weightedPrice)=214.67, min(weightedPrice)=16.66} EVENT! {max(weightedPrice)=134.02, min(weightedPrice)=70.26}

Question 5:

Extend the previous query so that it now also shows the difference between the minimum and maximum weighted price for each interval.

Thoughts:

Aggregate functions can be used to generate statistics which add value to the business.

Query:

SELECT MIN(weightedPrice), MAX(weightedPrice), MAX(weightedPrice) –

MIN(weightedPrice) AS Difference FROM StockTick.win:length_batch(100)

Result:

EVENT! {(max(weightedPrice)-min(weightedPrice))=0.3,max(weightedPrice)=0.36, min(weightedPrice)=0.06}

EVENT!{(max(weightedPrice)-min(weightedPrice))=0.8700000000000001,max(weightedPrice)=1.06, min(weightedPrice)=0.19}

EVENT! {(max(weightedPrice)-min(weightedPrice))=8.0,max(weightedPrice)=8.67, min(weightedPrice)=0.67}

EVENT!{(max(weightedPrice)-min(weightedPrice))=24.549999999999997,max(weightedPrice)=29.58, min(weightedPrice)=5.03}

EVENT!{(max(weightedPrice)-min(weightedPrice))=3.97,max(weightedPrice)=6.11, min(weightedPrice)=2.14} EVENT!{(max(weightedPrice)-min(weightedPrice))=3.26,max(weightedPrice)=7.05, min(weightedPrice)=3.79}

EVENT!{(max(weightedPrice)-min(weightedPrice))=1.9900000000000002,max(weightedPrice)=6.71, min(weightedPrice)=4.72}

EVENT!{(max(weightedPrice)-min(weightedPrice))=6.49,max(weightedPrice)=13.26, min(weightedPrice)=6.77}

EVENT!{(max(weightedPrice)-min(weightedPrice))=6.839999999999998,max(weightedPrice)=17.15, min(weightedPrice)=10.31}

EVENT!{(max(weightedPrice)-min(weightedPrice))=198.01,max(weightedPrice)=214.67, min(weightedPrice)=16.66}

EVENT!{(max(weightedPrice)-min(weightedPrice))=63.760000000000005,max(weightedPrice)=134.02, min(weightedPrice)=70.26}

Question 6:

Find the total weighted price of all stock tick events in each non-overlapping 100 tick interval of the BTC stock.

Thoughts:

We can also find total over a period of time or number of events.

Query:

SELECT SUM(weightedPrice) FROM StockTick.win:length_batch(100)

Result:

EVENT! {sum(weightedPrice)=9.17}

EVENT! {sum(weightedPrice)=39.289999999999864}

EVENT! {sum(weightedPrice)=230.47000000000017}

EVENT! {sum(weightedPrice)=1286.079999999999}

EVENT! {sum(weightedPrice)=358.8699999999993}

EVENT! {sum(weightedPrice)=523.9999999999995}

EVENT! {sum(weightedPrice)=540.7999999999997}

EVENT! {sum(weightedPrice)=1061.1}

EVENT! {sum(weightedPrice)=1265.1000000000008}

EVENT! {sum(weightedPrice)=6740.919999999999}

EVENT! {sum(weightedPrice)=10550.990000000007}

Question 7:

Find the dates and minimum weighted price for the BTC stock for each, non- overlapping 30 tick interval.

Thoughts:

Query:

SELECT timestamp as Date,weightedPrice as Weighted_Price FROM

StockTick.win:length_batch(30) where weightedPrice =(SELECT MIN(distinct weightedPrice)

FROM StockTick.win:length_batch(30))

Result:

EVENT! {Weighted_Price=0.06, Date=Sun Aug 01 00:00:00 CEST 2010} EVENT! {Weighted_Price=0.06, Date=Tue Aug 31 00:00:00 CEST 2010} EVENT! {Weighted_Price=0.06, Date=Thu Sep 30 00:00:00 CEST 2010} EVENT! {Weighted_Price=0.19, Date=Sat Oct 30 00:00:00 CEST 2010} EVENT! {Weighted_Price=0.19, Date=Fri Dec 10 00:00:00 CET 2010} EVENT! {Weighted_Price=0.29, Date=Wed Dec 29 00:00:00 CET 2010} EVENT! {Weighted_Price=0.43, Date=Fri Jan 28 00:00:00 CET 2011} EVENT! {Weighted_Price=0.75, Date=Mon Mar 21 00:00:00 CET 2011} EVENT! {Weighted_Price=0.67, Date=Tue Apr 05 00:00:00 CEST 2011} EVENT! {Weighted_Price=2.03, Date=Thu Apr 28 00:00:00 CEST 2011} EVENT! {Weighted_Price=8.3, Date=Sun May 29 00:00:00 CEST 2011} EVENT! {Weighted_Price=12.8, Date=Tue Jul 05 00:00:00 CEST 2011} EVENT! {Weighted_Price=7.72, Date=Sun Aug 07 00:00:00 CEST 2011} EVENT! {Weighted_Price=4.74, Date=Thu Sep 29 00:00:00 CEST 2011} EVENT! {Weighted_Price=2.28, Date=Wed Oct 19 00:00:00 CEST 2011} EVENT! {Weighted_Price=2.14, Date=Fri Nov 18 00:00:00 CET 2011} EVENT! {Weighted_Price=2.75, Date=Sun Dec 04 00:00:00 CET 2011} EVENT! {Weighted_Price=4.19, Date=Fri Dec 30 00:00:00 CET 2011} EVENT! {Weighted_Price=4.19, Date=Thu Feb 16 00:00:00 CET 2012} EVENT! {Weighted_Price=4.5, Date=Sun Mar 25 00:00:00 CET 2012} EVENT! {Weighted_Price=4.72, Date=Sun Apr 08 00:00:00 CEST 2012} EVENT! {Weighted_Price=4.93, Date=Fri May 11 00:00:00 CEST 2012} EVENT! {Weighted_Price=5.11, Date=Tue May 29 00:00:00 CEST 2012} EVENT! {Weighted_Price=6.51, Date=Wed Jul 04 00:00:00 CEST 2012} EVENT! {Weighted_Price=8.77, Date=Sun Jul 29 00:00:00 CEST 2012} EVENT! {Weighted_Price=10.0, Date=Sat Sep 01 00:00:00 CEST 2012} EVENT! {Weighted_Price=11.26, Date=Mon Oct 08 00:00:00 CEST 2012} EVENT! {Weighted_Price=10.31, Date=Fri Oct 26 00:00:00 CEST 2012} EVENT! {Weighted_Price=12.1, Date=Tue Nov 27 00:00:00 CET 2012} EVENT! {Weighted_Price=13.31, Date=Tue Dec 25 00:00:00 CET 2012}

EVENT! {Weighted_Price=16.66, Date=Fri Jan 25 00:00:00 CET 2013} EVENT! {Weighted_Price=29.05, Date=Sat Feb 23 00:00:00 CET 2013} EVENT! {Weighted_Price=65.33, Date=Tue Apr 16 00:00:00 CEST 2013} EVENT! {Weighted_Price=91.3, Date=Fri May 03 00:00:00 CEST 2013} EVENT! {Weighted_Price=98.01, Date=Sun Jun 09 00:00:00 CEST 2013} EVENT! {Weighted_Price=70.26, Date=Sat Jul 06 00:00:00 CEST 2013} EVENT! {Weighted_Price=91.45, Date=Mon Jul 22 00:00:00 CEST 2013}

Graph:

Question 8:

What are patterns and pattern operators? What does the every and followed-by(->) patterns do?

Explain the difference between the following: every A -> B

A -> every B every A -> every B every (A -> B)

Patterns are rules for matching events very similar to the where clause of SQL but they are applied before the events are included in current batch window.

Pattern operators are used to combine patterns (and, or) and order them (->). We can also have custom pattern operators.

Differences:

every A -> B: It means for every event A followed by event B. The pattern restarts whenever event A is encountered.

A -> every B: It means for event A followed by every event B. The pattern restarts whenever event B is encountered.

every A -> every B: It means for every event A followed by every event B. The pattern restarts on encountering both event A and B.

every (A -> B): It means for every event A followed by event B. The pattern restarts after encountering event B.

Question 9:

Find all the start and stop dates with volume, when the weighted price of the BTC stock increases more than 100 times and volume increases by 40%.

Thoughts:

We can track changes in data between events using patterns.

Query:

SELECT a.timestamp AS StartDate, b.timestamp AS StopDate,

a.volume AS StartVolume, b.volume AS StopVolume

FROM pattern[every a=StockTick -> every b=StockTick(b.weightedPrice > (100 * a.weightedPrice) AND b.volume > (1.4 * a.volume))]

Result:

EVENT! {StartDate=Sun Aug 01 00:00:00 IST 2010, StopVolume=64003.37, StopDate=Fri May 13 00:00:00 IST 2011, StartVolume=2601.0}

EVENT! {StartDate=Sun Aug 01 00:00:00 IST 2010, StopVolume=55390.45, StopDate=Sat May 14 00:00:00 IST 2011, StartVolume=2601.0}

EVENT! {StartDate=Sun Aug 01 00:00:00 IST 2010, StopVolume=18294.87, StopDate=Sun May 15 00:00:00 IST 2011, StartVolume=2601.0}

EVENT! {StartDate=Sun Aug 01 00:00:00 IST 2010, StopVolume=44141.41, StopDate=Mon May 16 00:00:00 IST 2011, StartVolume=2601.0}

EVENT! {StartDate=Sun Aug 01 00:00:00 IST 2010, StopVolume=35345.46, StopDate=Tue May 17 00:00:00 IST 2011, StartVolume=2601.0}

EVENT! {StartDate=Sun Aug 01 00:00:00 IST 2010, StopVolume=11556.09, StopDate=Wed Aug 28 00:00:00 IST 2013, StartVolume=2601.0}

EVENT! {StartDate=Sun Aug 01 00:00:00 IST 2010, StopVolume=7362.92, StopDate=Thu Aug 29 00:00:00 IST 2013, StartVolume=2601.0}

EVENT! {StartDate=Sun Aug 01 00:00:00 IST 2010, StopVolume=44989.75, StopDate=Fri Aug 30 00:00:00 IST 2013, StartVolume=2601.0}

EVENT! {StartDate=Sun Aug 01 00:00:00 IST 2010, StopVolume=30617.8, StopDate=Sat Aug 31 00:00:00 IST 2013, StartVolume=2601.0}

Question 10:

What is technical analysis? What is algorithmic trading?

Monitoring the market based on changes in price is called technical analysis. It is the study of supply and demand in the market in order to determine the trend or direction in future.

Algorithmic trading is use of advanced mathematical models for decision making in the market. It is used to optimize the time to place an order so that it makes minimal impact on the stock’s price.

Question 11:

Thoughts:

We can combine multiple pattern using pattern operators to get desired results.

Query:

SELECT b.timestamp AS BuyDate, b.close AS BuyClosingPrice, d.timestamp AS SellDate, d.close AS SellClosingPrice

FROM pattern[(every a=StockTick -> every b=StockTick(b.weightedPrice < a.weightedPrice AND b.volume >= 1.7 * a.volume))

OR (every c=StockTick -> every d=StockTick(d.weightedPrice >= 3 * c.weightedPrice AND d.volume >= 1.7 * c.volume))]

Result:

BuyDate=Mon Aug 16 00:00:00 IST 2010}

BuyDate=Sun Aug 22 00:00:00 IST 2010}

BuyDate=Fri Aug 27 00:00:00 IST 2010}

BuyDate=Sat Aug 28 00:00:00 IST 2010}

BuyDate=Sun Aug 29 00:00:00 IST 2010}

BuyDate=Mon Aug 30 00:00:00 IST 2010}

EVENT! {SellDate=Thu Aug 29 00:00:00 IST 2013, SellClosingPrice=129.30034,

BuyDate=Fri Aug 30 00:00:00 IST 2013}

EVENT! {SellDate=Sat Aug 31 00:00:00 IST 2013, SellClosingPrice=141.0,

Question 12:

Thoughts:

We can follow trend easily using Esper patterns.

Query:

f.timestamp AS SellDate, f.close AS SellClosingPrice

FROM pattern[(EVERY a=StockTick -> EVERY b=StockTick(b.weightedPrice > a.weightedPrice) -> EVERY c=StockTick(c.weightedPrice > b.weightedPrice AND c.close > a.close))

OR (EVERY d=StockTick -> EVERY e=StockTick(e.weightedPrice < d.weightedPrice) -> EVERY f=StockTick(f.weightedPrice < e.weightedPrice AND f.close > d.close))]

Result:

BuyDate=Fri Oct 08 00:00:00 IST 2010}

BuyDate=Sat Oct 09 00:00:00 IST 2010}

BuyDate=Sun Oct 10 00:00:00 IST 2010}

BuyDate=Thu Aug 29 00:00:00 IST 2013}

BuyDate=Fri Aug 30 00:00:00 IST 2013}

BuyDate=Sat Aug 31 00:00:00 IST 2013}

Question 13:

What are pattern guards? Why cannot the timer:within pattern guard be used in simulated environments? What alternatives exist to control time and dates?

Pattern guards are WHERE-conditions that control the lifecycle of subexpressions. It affects the overall result and can make the result better if used properly, They limit the scope of the pattern to another quanity like time or condition.timer:within pattern guard depends on actual latency between events but in simulated environments the events occur in quick succession and hence timer:within cannot be used.

For example StockTick where timer.within(120 sec)

we use the timer:within in a crucial system and set the timer to 2 Minutes, and the data is coming continuously, then the system will stop processing the data after 2 minutes which is not a good practice, because the data that comes after two minute might be of much more importance.

We can use the WHERE clause of EPL to control time and date. Also WHILE and guard time interval expressions can be used.

For example:every a=A -> b=B where timer:within (a.delta seconds)We can use the WHERE clause of EPL to control time and date.

Question 14:

Find the start date, stop date and weighted prices when the BTC stock value has decreased by three times within a 15 day period,

Thoughts:

We can analyze events based on trend in a fixed size time of length window.

Query:

SELECT a.weightedPrice,b.weightedPrice FROM pattern [every a=StockTick-> b=StockTick(weightedPrice * 3 < a.weightedPrice AND timestamp.after(a.timestamp,15 days))]

Result:

EVENT! {a.weightedPrice=27.25, b.weightedPrice=7.82}

EVENT! {a.weightedPrice=16.41, b.weightedPrice=5.31}

EVENT! {a.weightedPrice=15.51, b.weightedPrice=5.03}

EVENT! {a.weightedPrice=15.08, b.weightedPrice=4.98}

EVENT! {a.weightedPrice=14.48, b.weightedPrice=4.82}

EVENT! {a.weightedPrice=14.35, b.weightedPrice=4.77}

EVENT! {a.weightedPrice=14.15, b.weightedPrice=4.69}

EVENT! {a.weightedPrice=13.08, b.weightedPrice=4.29}

EVENT! {a.weightedPrice=12.8, b.weightedPrice=4.03}

EVENT! {a.weightedPrice=11.3, b.weightedPrice=3.64}

EVENT! {a.weightedPrice=8.67, b.weightedPrice=2.85}

EVENT! {a.weightedPrice=7.98, b.weightedPrice=2.62}

EVENT! {a.weightedPrice=7.48, b.weightedPrice=2.28}

EVENT! {a.weightedPrice=6.74, b.weightedPrice=2.23}

EVENT! {a.weightedPrice=6.47, b.weightedPrice=2.14}

EVENT! {a.weightedPrice=214.67, b.weightedPrice=71.11}

Question 15:

What are variables in Esper and how can they be used to hold and change state? Give examples!

Just as any other programming language a variable in Esper is an object, event or scalar value which is available for use in all statements including patterns after declaration. It can be used in patterns and other statements including output clauses. They have to be declared or configured before use by defining each variables type and name.For example to create a variable we can use create variable syntax. Variables can be assigned new values by using the on set syntax via the setVariableValue methods on EPRuntime.

For example:create [constant] variable variable_type variable_name;where variable_type is one of string, char, character, bool, short, int, long, double, float among others

Question 16:

Write your own trading algorithm and use it to find buy and sell dates. No hard- coding of dates or numbers are allowed! Use the knowledge you have on Esper to write your query of choice. If you want to change the Java files, you are allowed to, but you do not have to (e.g. needed for multiple queries).

Thoughts:

My trading algorithm is as follows:

We sell when the weighted price has risen for three days (not necessarily adjacent) and the volume has increased by 35% between first and third day.

We buy when the weighted price has fallen for three days (again not necessarily adjacent) and the volume has decreased by 45% between first and third day.

Query:

SELECT c.timestamp AS BuyDate, c.close AS BuyClosingPrice, f.timestamp AS SellDate, f.close AS SellClosingPrice

FROM pattern[(EVERY a=StockTick -> EVERY b=StockTick(b.weightedPrice > a.weightedPrice) -> EVERY c=StockTick(c.weightedPrice > b.weightedPrice AND c.volume > 0.55 * a.volume))

OR (EVERY d=StockTick -> EVERY e=StockTick(e.weightedPrice < d.weightedPrice) -> EVERY f=StockTick(f.weightedPrice < e.weightedPrice AND f.volume < 0.65 * d.volume))]

Result:

BuyDate=Fri Oct 08 00:00:00 IST 2010}

BuyDate=Sat Oct 09 00:00:00 IST 2010}

BuyDate=Sun Oct 10 00:00:00 IST 2010}

BuyDate=Thu Aug 29 00:00:00 IST 2013}

BuyDate=Fri Aug 30 00:00:00 IST 2013}

BuyDate=Sat Aug 31 00:00:00 IST 2013}

Question 17:

What did you struggle with during this exercise? Was it the provided code’s fault, Esper’s or your own fault? What was different from working with regular database systems?

I struggled with the syntax and reverse flow of data during this excercise. The patterns and batch window concepts were new and after solving the problems seemed like essential to the philosophy of Complex Event Processing.

It was not anyone’s fault but just a learning curve and forgoing many deep rooted practices like getting a single result for a typical query like count(*).

In regular database system we mostly have data at a state and we run the queries whereas in Esper the queries are know beforehand and the data comes as and when.

NB: Some of the results are too lengthy to be written here in this file so I just

wrote the beginning and ending part of the results, the complete result can be found in the zip file.

### David Marks

Dear sir/Ma'am

Our dedication and hard work towards developing quality content has made us competent to provide excellent services to the clients as per their needs. We ensure plagiarism free writing obeying academic integrity and honesty making us a dedicated team towards developing original content helping students to ensure high grades. Our services are hassle free, timely yet at an affordable rate driving clients to avail it for a longer period of time making them loyal and satisfied towards us.

Services we offer:
Essay writing
Report writing
Assignment writing
Reflective writing
Proposal writing
Dissertation writing
Thesis writing
CDR writing
Annotated bibliography
PowerPoint presentation
Article/book review
Why us?
Plagiarism free original content
Experienced writers
Plagiarism and Grammarly report
Dedicate Team Leaders and Quality Checkers
Subjects we cover:
Marketing management
Human resource management
Literature
Law
Finance
Accounts
Economics
Nursing
Sociology
Environmental science
Political science
History
Journalism and Masscommunication
Geography

Declaration: Working with us will give you the opportunity to avail divergent range of academic services at affordable rates in assistance with the dedicated team having members from different disciplines holding high degrees in their respective domains. We are experienced in developing B-plan, writing dissertations and theses having employed highly qualified and experienced writers.

You can reach us at-
Email:- ozpaperhelp@gmail.com
https://www.ozpaperhelp.com/
https://www.cheapassignmenthelp.co.uk
https://www.freeassignmenthelp.com
https://cheapassignmenthelp.blogspot.com/
Thanks
Oz Paper Help

https://www.ozpaperhelp.com/
1 Step 1
GET INSTANT ASSIGNMENT HELP
keyboard_arrow_leftPrevious
Nextkeyboard_arrow_right