Mary Ann Davidson Blog

No, You Really Cant

By User701213-Oracle on Aug 10, 2015

I have been doing a lot of writing recently. Some of my writing has been with my sister, with whom
I write murder mysteries using the nom-de-plume Maddi Davidson. Recently, weve been working
on short stories, developing a lot of fun new ideas for dispatching people (literarily speaking,
though I think about practical applications occasionally when someone tailgates me).

Writing mysteries is a lot more fun than the other type of writing Ive been doing. Recently, I have
seen a large-ish uptick in customers reverse engineering our code to attempt to find security
vulnerabilities in it. <Insert big sigh here.> This is why Ive been writing a lot of letters to
customers that start with hi, howzit, aloha but end with please comply with your license
agreement and stop reverse engineering our code, already.

I can understand that in a world where it seems almost every day someone else had a data breach
and lost umpteen gazillion records to unnamed intruders who may have been working at the behest
of a hostile nation-state, people want to go the extra mile to secure their systems. That said, you
would think that before gearing up to run that extra mile, customers would already have ensured
theyve identified their critical systems, encrypted sensitive data, applied all relevant patches, be on
a supported product release, use tools to ensure configurations are locked down in short, the usual
security hygiene before they attempt to find zero day vulnerabilities in the products they are
using. And in fact, there are a lot of data breaches that would be prevented by doing all that stuff, as
unsexy as it is, instead of hyperventilating that the Big Bad Advanced Persistent Threat using a
zero-day is out to get me! Whether you are running your own IT show or a cloud provider is
running it for you, there are a host of good security practices that are well worth doing.

Even if you want to have reasonable certainty that suppliers take reasonable care in how they build
their products and there is so much more to assurance than running a scanning tool - there are a lot
of things a customer can do like, gosh, actually talking to suppliers about their assurance programs
or checking certifications for products for which there are Good Housekeeping seals for (or good
code seals) like Common Criteria certifications or FIPS-140 certifications. Most vendors at least,
most of the large-ish ones I know have fairly robust assurance programs now (we know this
because we all compare notes at conferences). Thats all well and good, is appropriate customer due
diligence and stops well short of hey, I think I will do the vendors job for him/her/it and look for
problems in source code myself, even though:

 A customer cant analyze the code to see whether there is a control that prevents the attack
the scanning tool is screaming about (which is most likely a false positive)
A customer cant produce a patch for the problem only the vendor can do that
A customer is almost certainly violating the license agreement by using a tool that does
static analysis (which operates against source code)

I should state at the outset that in some cases I think the customers doing reverse engineering are
not always aware of what is happening because the actual work is being done by a consultant, who
runs a tool that reverse engineers the code, gets a big fat printout, drops it on the customer, who
then sends it to us. Now, I should note that we dont just accept scan reports as proof that there is a
there, there, in part because whether you are talking static or dynamic analysis, a scan report is not
proof of an actual vulnerability. Often, they are not much more than a pile of steaming FUD.
(That is what I planned on saying all along: FUD.) This is why we require customers to log a
service request for each alleged issue (not just hand us a report) and provide a proof of concept
(which some tools can generate).

If we determine as part of our analysis that scan results could only have come from reverse
engineering (in at least one case, because the report said, cleverly enough, static analysis of Oracle
XXXXXX), we send a letter to the sinning customer, and a different letter to the sinning
consultant-acting-on-customers behalf reminding them of the terms of the Oracle license
agreement that preclude reverse engineering, So Please Stop It Already. (In legalese, of course. The
Oracle license agreement has a provision such as: "Customer may not reverse engineer,
disassemble, decompile, or otherwise attempt to derive the source code of the Programs..." which
we quote in our missive to the customer.) Oh, and we require customers/consultants to destroy the
results of such reverse engineering and confirm they have done so.

Why am I bringing this up? The main reason is that, when I see a spike in X, I try to get ahead of it.
I dont want more rounds of you broke the license agreement, no, we didnt, yes, you did, no,
we didnt. Id rather spend my time, and my teams time, working on helping development
improve our code than argue with people about where the license agreement lines are.

Now is a good time to reiterate that Im not beating people up over this merely because of the
license agreement. More like, I do not need you to analyze the code since we already do that, its
our job to do that, we are pretty good at it, we can unlike a third party or a tool actually analyze
the code to determine whats happening and at any rate most of these tools have a close to 100%
false positive rate so please do not waste our time on reporting little green men in our code. I am
not running away from our responsibilities to customers, merely trying to avoid a painful, annoying,
and mutually-time wasting exercise.

For this reason, I want to explain what Oracles purpose is in enforcing our license agreement (as it
pertains to reverse engineering) and, in a reasonably precise yet hand-wavy way, explain where the
line is you cant cross or you will get a strongly-worded letter from us. Caveat: I am not a lawyer,
even if I can use words like stare decisis in random conversations. (Except with my dog, because he
only understands Hawaiian, not Latin.) Ergo, when in doubt, refer to your Oracle license agreement,
which trumps anything I say herein!

With that in mind, a few FAQ-ish explanations:

Q. What is reverse engineering?

A. Generally, our code is shipped in compiled (executable) form (yes, I know that some code is
interpreted). Customers get code that runs, not the code as written. That is for multiple reasons
such as users generally only need to run code, not understand how it all gets put together, and the
fact that our source code is highly valuable intellectual property (which is why we have a lot of
restrictions on who accesses it and protections around it). The Oracle license agreement limits what
you can do with the as-shipped code and that limitation includes the fact that you arent allowed to
de-compile, dis-assemble, de-obfuscate or otherwise try to get source code back from executable
code. There are a few caveats around that prohibition but there isnt an out for unless you are
looking for security vulnerabilities in which case, no problem-o, mon!

If you are trying to get the code in a different form from the way we shipped it to you as in, the
way we wrote it before we did something to it to get it in the form you are executing, you are
probably reverse engineering. Dont. Just dont.

Q. What is Oracles policy in regards to the submission of security vulnerabilities (found by tools or
not)?

A. We require customers to open a service request (one per vulnerability) and provide a test case to
verify that the alleged vulnerability is exploitable. The purpose of this policy is to try to weed out
the very large number of inaccurate findings by security tools (false positives).

Q. Why are you going after consultants the customer hired? The consultant didnt sign the license
agreement!

A. The customer signed the Oracle license agreement, and the consultant hired by the customer is
thus bound by the customers signed license agreement. Otherwise everyone would hire a

consultant to say (legal terms follow) Nanny, nanny boo boo, big bad consultant can do X even if
the customer cant!

Q. What does Oracle do if there is an actual security vulnerability?

A. I almost hate to answer this question because I want to reiterate that customers Should Not and
Must Not reverse engineer our code. However, if there is an actual security vulnerability, we will fix
it. We may not like how it was found but we arent going to ignore a real problem that would be a
disservice to our customers. We will, however, fix it to protect all our customers, meaning
everybody will get the fix at the same time. However, we will not give a customer reporting such an
issue (that they found through reverse engineering) a special (one-off) patch for the problem. We
will also not provide credit in any advisories we might issue. You cant really expect us to say
thank you for breaking the license agreement.

Q. But the tools that decompile products are getting better and easier to use, so reverse engineering
will be OK in the future, right?

A. Ah, no. The point of our prohibition against reverse engineering is intellectual property
protection, not how can we cleverly prevent customers from finding security vulnerabilities
bwahahahaha so we never have to fix them bwahahahaha. Customers are welcome to use tools
that operate on executable code but that do not reverse engineer code. To that point, customers
using a third party tool or service offering would be well-served by asking questions of the tool (or
tool service) provider as to a) how their tool works and b) whether they perform reverse engineering
to do what they do. An ounce of discussion is worth a pound of no we didnt, yes you did,
didnt, did arguments. *

Q. But I hired a really cool code consultant/third party code scanner/whatever. Why wont mean
old Oracle accept my scan results and analyze all 400 pages of the scan report?

A. Hoo-boy. I think I have repeated this so much it should be a song chorus in a really annoying hip
hop piece but here goes: Oracle runs static analysis tools ourselves (heck, we make them), many of
these goldurn tools are ridiculously inaccurate (sometimes the false positive rate is 100% or close to
it), running a tool is nothing, the ability to analyze results is everything, and so on and so forth. We
put the burden on customers or their consultants to prove there is a There, There because otherwise,
we waste a boatload of time analyzing nothing** when we
could be spending those resources, say, fixing actual security vulnerabilities.

Q. But one of the issues I found was an actual security vulnerability so that justifies reverse

engineering, right?

A. Sigh. At the risk of being repetitive, no, it doesnt, just like you cant break into a house because
someone left a window or door unlocked. Id like to tell you that we run every tool ever developed
against every line of code we ever wrote, but thats not true. We do require development teams (on
premises, cloud and internal development organizations) to use security vulnerability-finding tools,
weve had a significant uptick in tools usage over the last few years (our metrics show this) and we
do track tools usage as part of Oracle Software Security Assurance program. We beat up I mean,
require development teams to use tools because it is very much in our interests (and customers
interests) to find and fix problems earlier rather than later.

That said, no tool finds everything. No two tools find everything. We dont claim to find everything.
That fact still doesnt justify a customer reverse engineering our code to attempt to find
vulnerabilities, especially when the key to whether a suspected vulnerability is an actual
vulnerability is the capability to analyze the actual source code, which frankly hardly any third
party will be able to do, another reason not to accept random scan reports that resulted from reverse
engineering at face value, as if we needed one.

Q. Hey, Ive got an idea, why not do a bug bounty? Pay third parties to find this stuff!

A. <Bigger sigh.> Bug bounties are the new boy band (nicely alliterative, no?) Many companies are
screaming, fainting, and throwing underwear at security researchers**** to find problems in their
code and insisting that This Is The Way, Walk In It: if you are not doing bug bounties, your code
isnt secure. Ah, well, we find 87% of security vulnerabilities ourselves, security researchers find
about 3% and the rest are found by customers. (Small digression: I was busting my buttons today
when I found out that a well-known security researcher in a particular area of technology reported a
bunch of alleged security issues to us except we had already found all of them and we were
already working on or had fixes. Woo hoo!)

I am not dissing bug bounties, just noting that on a strictly economic basis, why would I throw a lot
of money at 3% of the problem (and without learning lessons from what you find, it really is
whack a code mole) when I could spend that money on better prevention like, oh, hiring another
employee to do ethical hacking, who could develop a really good tool we use to automate finding
certain types of issues, and so on. This is one of those full immersion baptism or sprinkle water
over the forehead issues we will allow for different religious traditions and do it OUR way and
others can do it THEIR way. Pax vobiscum.

Q. If you dont let customers reverse engineer code, they wont buy anything else from you.

A. I actually heard this from a customer. It was ironic because in order for them to buy more
products from us (or use a cloud service offering), theyd have to sign a license agreement! With
the same terms that the customer had already admitted violating. Honey, if you wont let me cheat
on you again, our marriage is through. Ah, er, you already violated the forsaking all others part
of the marriage vow so I think the marriage is already over.

The better discussion to have with a customer and I always offer this is for us to explain what
we do to build assurance into our products, including how we use vulnerability finding tools. I want
customers to have confidence in our products and services, not just drop a letter on them.

Q. Surely the bad guys and some nations do reverse engineer Oracles code and dont care about
your licensing agreement, so why would you try to restrict the behavior of customers with good
motives?

A. Oracles license agreement exists to protect our intellectual property. Good motives and
given the errata of third party attempts to scan code the quotation marks are quite apropos are not
an acceptable excuse for violating an agreement willingly entered into. Any more than but
everybody else is cheating on his or her spouse is an acceptable excuse for violating forsaking all
others if you said it in front of witnesses.

At this point, I think I am beating a dead or should I say, decompiled horse. We ask that
customers not reverse engineer our code to find suspected security issues: we have source code, we
run tools against the source code (as well as against executable code), its actually our job to do that,
we dont need or want a customer or random third party to reverse engineer our code to find
security vulnerabilities. And last, but really first, the Oracle license agreement prohibits it. Please
dont go there.

* I suspect at least part of the anger of customers in these back-and-forth discussions is because the
customer had already paid a security consultant to do the work. They are angry with us for having
been sold a bill of goods by their consultant (where the consultant broke the license agreement).

** The only analogy I can come up with is my bookshelf. Someone convinced that I had a
prurient interest in pornography could look at the titles on my bookshelf, conclude they are
salacious, and demand an explanation from me as to why I have a collection of steamy books. For
example (these are all real titles on my shelf):

1.
2.
3.
4.

Thunder Below! (whoo boy, must be hot stuff!)

Naked Economics (nude Keynesians!)***
Inferno (even hotter stuff!)
At Dawn We Slept (you must be exhausted from your, ah, nighttime activities)

My response is that I dont have to explain my book tastes or respond to baseless FUD. (If anybody
is interested, the actual book subjects are, in order, 1) the exploits of WWII submarine skipper and
Congressional Medal of Honor recipient CAPT Eugene Fluckey, USN 2) a book on economics 3) a
book about the European theater in WWII and 4) the definitive work concerning the attack on Pearl
Harbor. )

*** Absolutely not, I loathe Keynes. There are more extant dodos than actual Keynesian
multipliers. Although dodos and true believers in Keynesian multipliers are interchangeable
terms as far as I am concerned.

**** I might be exaggerating here. But maybe not.