Introducing Lua

Introducing Lua for MySQL Proxy programming
Giuseppe Maxia QA Developer December 2007

Copyright 2004-2007 MySQL AB The Worlds Most Popular Open Source Database
About me
Giuseppe Maxia MySQL QA Developer a.k.a. The Data Charmer Despite popular belief, not a Lua expert (less than six months experience)
Agenda
MySQL Proxy architecture (very quickly) Introducing Lua Language fundamentals Examples Comments
Copyright 2004-2007 MySQL AB
The Worlds Most Popular Open Source Database
References
MySQL Forge
http://forge.mysql.com/wiki/MySQL_Proxy
Getting started article

http://dev.mysql.com/tech-resources/articles/proxygettingstarted.html
Lua
http://www.lua.org/docs.html http://lua-users.org/wiki/TutorialDirectory
Proxy (< lat. procuratio) = Someone taking care of someone else's interests A server proxy is something acting on behalf of another server
Proxy Overview
Proxy Overview
PROXY CORE connection hook read query hook read result hook Lua script function function function function function
Lua Overview
??
Why not ...
Perl ? PHP? Javascript? [whatever]?
Lua Overview
SMALL ( < 200 KB) DESIGNED for
EMBEDDED systems Widely used (lighttpd) lighttpd, like MySQL Proxy, was created by Jan Kneschke
Lua Overview
Very popular among game writers
10
Proxy Overview
global context
session session context session context session context session context session context context Lua script
connect_server read_handshake read_auth read_auth_result read_query read_query_result disconnect_client
session context
11
Using Lua les

/usr/local/sbin/mysql-proxy \ --proxy-lua-script=/path/name.lua IMPORTANT! THE SCRIPT DOES NOT START UNTIL THE FIRST CLIENT CONNECTION
12
Lua types
nil number string table function userdata
a b c t f u = = = = = = nil 1 'abc' { a,b,c } print some_C_struct
13
Lua comments
-- simple comment print(1) --[[ print(2) print('hello') --]] print(3)
14
Lua comments
-- simple comment --[=[ print(1) --[[ print(2) print('hello') --]] print(3) --]=]
15
numbers and strings

nil = no value (false) number = oating point a = nil '5' =/= 5
b = 5; c = '5' print (b == c) false print (b == c +0) true
16
numbers and strings
conversion on demand a = 5 ; b = '5' print(type(a), type(b)) number string print(type(b+0)) number print(type(a .. "")) string
17
numbers and strings
conversion on demand a = 5 ; b = '5' print(type(tostring(a))) string print(type(tonumber(b))) number
18
strings
smart quoting
a b c d e = = = = = 'Hello' "World" "Can't" [[Don't say "Hello"]] [=["d'oh" [[braces]]!]=]
19
tables
associative arrays can be used as arrays can create complex
structures t1 = {10, 20, 30 } t2 = { a = 'abc', b = 2, c = { 3, 4} }
20
functions
can be assigned to
variables new functions can override existing ones function f (x) print(x) end g = f g(10)
21
userdata
containers to exchange
data between Lua and host language can have "tag methods"
22
Statements
normal assignments
a = 3 b = get_num() -- func return multiple assignments a,b = 3,2 multiple return values function x () return 1, 'OK' end a, b, c = x() -- a = 1, b = 'OK', c = nil
23
statement blocks
if
if condition then statements end while while condition do statements end
24
statement blocks
for
for var = 1, 10 [,step] do statements end for n,v in pairs(table_var) do statements end
for
25
sample function
read_query
1 function read_query(packet) 2 if packet:byte() ~= 3 proxy.PROXY_COM_QUERY 4 then 5 return 6 end 7 local query = packet:sub(2) 8 print('received ' .. query) 9 end
26
some details
== equals ~= not equal string.byte(packet) packet:byte() string.sub(packet,2) packet:sub(2) 'abc' .. '123'
== 'abc123'
27
Using tables
t = {} t[1] = 'a' --First element 1, ! 0 t[2] = 'b' table.insert(t, 'c') -- or t[ #t +1 ] = 'c' t = {'a', 'b', 'c' } t = {1 = 'a', 2 = 'b', 3 = 'c'} print (t[2]) b
28
Using tables
sizes = {} sizes['john'] = 'XL' sizes['paul'] = 'M' sizes['fred'] = 'L' sizes = { john = 'XL', paul = 'M', fred = 'L', }
29
Using tables
sizes = { john = 'XL', paul = 'M', fred = 'L', } print(sizes['john']) XL print(sizes.paul) M
30
Using tables
for i, v in ipairs(t) do print (i ..' -> ' .. v) end for name,size in pairs(sizes) do print(name .. ' ' .. wears .. ' ' .. size) end
31
Watch out
/* C / C++ */ int a = 0; printf("%s\n", a ? "true" : "false"); false -- Lua a = 0 print ( a and "true" or "false") true
32
Watch out
-- Lua a = false print ( a and "true" or "false") false a = nil print ( a and "true" or "false") false
33
Finding text
query = 'SELECT id FROM t1' local cmd, column = query:match("(SELECT)%s+(%w+)") if cmd then -- do something with query end
34
Finding text
Regular expressions similar to Perl/PHP, but simpler
% instead of \ (captures) [character classes] ^ $ + - ? * no alternation (a|b) no modifiers /i
35
Finding text (Proxy way)

local tk = require('proxy.tokenizer') local tokens = tk.tokenize(query) if tokens[1].token_name == 'TK_SQL_SELECT' then -- do something with query end
36
Finding text (Proxy way)

-- each token is a table token = { token_name = 'TK_SQL_SELECT', text = 'select', token_id = 204 }
37
I/O
-- files are objects local fname = '/tmp/test.txt' assert(fh = io.open(fname,'r')) for line in fh:lines() do print(line) end fh:close()
38
I/O
-- files are objects local fname = '/tmp/test.txt' assert(fh = io.open(fname,'w')) for x = 1, 100 do fh:write('new row ' .. x) fh:flush() end fh:close()
39
Examples
http://forge.mysql.com/wiki/Lua_Scripts_For_MySQL_Proxy_Examples
all hooks session bandwidth user bandwidth blocking commands
40
all-hooks (1)
source: all-hooks.lua
function ... function ... function ... function ... function ... function ... function ... read_auth( auth ) connect_server() read_handshake( auth ) read_auth_result( auth ) disconnect_client() read_query (packet) read_query_result (inj)
41
all-hooks (2)
local access_ndx = 0 function print_access(msg) access_ndx = access_ndx + 1 print( string.format('%3d %-30s',access_ndx,msg)) end function read_auth( auth ) print_access ('inside read_auth ') end function connect_server() print_access ('inside connect_server') end function read_handshake( auth ) print_access ('inside read_handshake' ) end -- ... to be continued
42
all-hooks (3)
function read_auth_result( auth ) print_access ('inside read_auth_result ') end function read_query (packet) print_access('inside read_query \t' .. packet:sub(2)) proxy.queries:append(1, packet) return proxy.PROXY_SEND_QUERY end function read_query_result (inj) print_access('inside read_query_result \t' .. inj.query) end function disconnect_client() print_access('inside disconnect_client') end
43
read_query & read_query_result

client
query
result
function read_query
function read_query_result
MySQL Proxy
query result
SERVER
if a query is passed directly to the server, its result is NOT evaluated by read_query_result
44
read_query & read_query_result

client
query
result
function read_query_result function read_query query queue

query query query
MySQL Proxy
result
only if a query is added to the query queue, its result is evaluated by read_query_result
SERVER
45
all-hooks (4)
sample output /usr/local/sbin/mysql-proxy --proxy-lua-script=all-hooks.lua 1 inside connect_server 2 inside read_handshake 3 inside read_auth 4 inside read_auth_result 5 inside read_query select @@version_comment limit 1 6 inside read_query_result select @@version_comment limit 1 7 inside read_query select now() 8 inside read_query_result select now() 9 inside read_query 10 inside disconnect_client
46
session-bandwidth (1)
source: session-bandwidth.lua
local bandwidth = 0 function read_query (packet ) bandwidth = bandwidth + packet:len() proxy.queries:append(1, packet ) return proxy.PROXY_SEND_QUERY end -- to be continued ...
47
session-bandwidth (2)
-- (continue) function read_query_result(inj) local fields = inj.resultset.fields if (fields) then for i = 1, #fields do bandwidth = bandwidth + (fields[i] and fields[i].name:len() or 0) end for row in inj.resultset.rows do for i = 1, #inj.resultset.fields do bandwidth = bandwidth + (row[i] and row[i]:len() or 0) end end end print (proxy.connection.server['thread_id'],bandwidth) end
48
session-bandwidth
sample output: /usr/local/sbin/mysql-proxy --proxy-lua-script=bandwidth.lua 64 78 64 115 65 65 66 66 78 115 78 1118
49
user-bandwidth (1)
source: user-bandwidth.lua
proxy.global.bandwidth = proxy.global.bandwidth or {} local session_user function read_auth( auth ) session_user = auth.username proxy.global.bandwidth[session_user] = proxy.global.bandwidth[session_user] or 0 end -- to be continued ...
50
user-bandwidth (2)
function read_query (packet ) proxy.global.bandwidth[session_user ] = proxy.global.bandwidth[session_user] + packet:len() proxy.queries:append(1, packet ) return proxy.PROXY_SEND_QUERY end -- to be continued ...
51
user-bandwidth (3)
function read_query_result(inj) local fields = inj.resultset.fields local rows = inj.resultset.rows if fields then for i = 1, #fields do proxy.global.bandwidth[session_user] = proxy.global.bandwidth[session_user] + (fields[i] and fields[i].name:len() or 0) end if rows then for row in rows do for i = 1, #fields do proxy.global.bandwidth[session_user] = proxy.global.bandwidth[session_user] + (row[i] and row[i]:len() or 0) end end end end print (session_user .. ' -> ' .. proxy.global.bandwidth[session_user]) end
52
user-bandwidth (4)
Sample session gmax -> 376 gmax -> 399 root -> 599 root -> 10641 gmax -> 478 gmax -> 20520
53
query-block (1)
source: query-block.lua
local tokenizer = require("proxy.tokenizer") ---- make_regexp_from_command() --- creates a regular expression for fast scanning of the command --- @param cmd the command to be converted to regexp -function make_regexp_from_command(cmd) local regexp= '^%s*'; for ch in cmd:gmatch('(.)') do regexp = regexp .. '[' .. ch:upper() .. ch:lower() .. ']' end return regexp end -- to be continued ...
54
query-block (2)
local SHOW_REGEXP = make_regexp_from_command('show')
queries_to_filter = { { prefix = SHOW_REGEXP, keywords = { 'SHOW', 'TABLE', 'STATUS'} , }, { prefix = SHOW_REGEXP, keywords = { 'SHOW', 'TABLES'} , }, } -- to be continued ...
55
query-block (3)
function error_result (msg) proxy.response = { type = proxy.MYSQLD_PACKET_ERR, errmsg = msg, errcode = 7777, sqlstate = 'X7777', } return proxy.PROXY_SEND_RESULT end -- to be continued ...
56
query-block (4)
function read_query( packet ) if (packet:byte() ~= proxy.COM_QUERY) then return end local query = packet:sub(2) for i,query_to_filter in pairs(queries_to_filter) do if query:match(query_to_filter.prefix) then local full_tokens = tokenizer.tokenize(query) local tokens = tokenizer.bare_tokens(full_tokens, true) local found = 0 local requested = #query_to_filter.keywords for j,keyword in pairs(query_to_filter.keywords) do for k, token in pairs(tokens) do if token:upper() == keyword then found = found + 1 break end end end -- to be continued ...
57
query-block (5)
function read_query( packet ) -- ... if found == requested then -- to be filtered off return error_result('command <' .. table.concat(query_to_filter.keywords,' ') .. '> is not allowed' ) end
end
end
end
58
query-block (6)
sample output: show tables; ERROR 7777 (X7777): command <SHOW TABLES> is not allowed
59
Read more
http://www.lua.org/docs.html
online Lua documentation
60
Read more
http://www.inf.puc-rio.br/~roberto/pil2/
61
Read more
http://www.wrox.com/WileyCDA/WroxTitle/productCd-0470069171.html
62
QUESTIONS
?
Giuseppe Maxia QA Developer December 2007

Introducing Lua

Încărcat de

Informații document

Descriere originală:

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Introducing Lua

Încărcat de

Drepturi de autor:

Formate disponibile

Introducing Lua for MySQL Proxy programming

Giuseppe Maxia QA Developer December 2007

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

Getting started article

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

Perl ? PHP? Javascript? [whatever]?

The Worlds Most Popular Open Source Database

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

connect_server read_handshake read_auth read_auth_result read_query read_query_result disconnect_client

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

Using Lua les

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

numbers and strings

numbers and strings

numbers and strings

conversion on demand a = 5 ; b = '5' print(type(tostring(a))) string print(type(tonumber(b))) number

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

Copyright 2004-2007 MySQL AB

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

Copyright 2004-2007 MySQL AB

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

Copyright 2004-2007 MySQL AB

Finding text (Proxy way)

Finding text (Proxy way)

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

all hooks session bandwidth user bandwidth blocking commands

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

read_query & read_query_result

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

read_query & read_query_result

function read_query_result function read_query query queue

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

Copyright 2004-2007 MySQL AB

The Worlds Most Popular Open Source Database

Copyright 2004-2007 MySQL AB