trustedsignal -- blog

Check the uids and gids

2011-12-26T23:23:00.001-06:00

While working on body-outliers, the Python script I wrote to do statistical analysis on fls bodyfiles in an effort to find malicious files in compromised file systems, one of the things I was ignoring completely, but that stuck out like a sore thumb when reviewing the data, was user and group IDs for files in Unix and Linux file systems.

When attackers build their kits that they intend to drop on remote hosts as backdoors, packet sniffers, key loggers, etc., they often use tar and gzip to create compressed archives of those files, then they can issue a command like wget to download the archive to the compromised host where they will "untar" the archive and move their malicious binaries into desired paths on the system.

One of the "features" of tar, as the manpage tells us is, "by default, newly-created files are owned by the user running tar." This means that if the attacker is logged into his own system as a non-root user and he's are compiling binaries which will replace legitimate binaries on the target system, those binaries will retain his user and group id information when they are tar'd up. Of course a careful, thoughtful attacker can take a variety of countermeasures to change this, but many are not so careful.

As a result, when they install malicious code on target systems, there's a chance those binaries will be installed with user IDs and group IDs (henceforth uid and gid) that don't match other files in those locations. These are obvious outliers and as I was working on the next version of body-outliers, I had written the code to calculate the average uid and gid values on a per directory basis, then calculate standard deviation, then alert on the outliers, but this sort of statistical analysis didn't make sense for uids and gids, because for the most part, they are uniform throughout the file system, with a few exceptions like /tmp, /var/spool/cron, /var/spool/mail and many custom software packages, but many system directories like /dev, /bin, /usr, etc. are set uid and gid 0, meaning the files are owned by the root account and belong to the root group. In this context, standard deviation didn't make much sense, so I modified my code to do another form of statistical analysis; namely calculating distributions.

Calculating distributions is just fancy talk for counting the occurrences of a thing, say, how many files are uid 0, how many are uid 1000, and so on, then displaying this information. This type of analysis lends itself well to finding oddball uid and gid files in compromised *nix file systems. On the hacked system I spoke of during my SECTor 2011 talk (video, slides), finding these unusual uid and gid files correlates very well to finding attacker code for precisely the reasons described above.

Here's a sample run of the script, which I'm calling body-ugid-dist.py, run against the same bodyfile as the one in the SECTor talk, this has been trimmed down a bit:

./body-ugid-dist.py --file sda1_bodyfile.txt --meta uid
[+] Checking command line arguments.
[+] sda1_bodyfile.txt may be a bodyfile.
[+] Discarded 0 files named .. or .
[+] Discarded 0 bad lines from sda1_bodyfile.txt.
[+] Added 20268 paths to meta.

...

Path:  /etc/cron.daily
==========================
Count:       1  uid:  1000
Count:       9  uid:     0

...

Path:  /usr/lib
==========================
Count:       1  uid:    10
Count:       1  uid:    37
Count:       1  uid:  1000
Count:    2082  uid:     0

...

In actuality this script returns 499 lines of output, representing about 350 "Counts," most of which were specific to the custom application running on the system. But the overall bodyfile had more than 200 thousand lines, so this is a considerable reduction in data, which is vital to any investigation. What the above output tells us is that of the 10 files in /etc/cron.daily, nine of them are uid 0 and one is uid 1000, that's a lead that may be worth pursuing and indeed, in this case, it is malicious code. The next entry shows tht /usr/lib contains 2085 files with 2082 of them being uid 0 and three others that are one offs and certainly worth looking into. In that case, two of the three are malicious code.

body-ugid-dist.py is available from my github repo. Unfortunately, it's only going to be useful for *nix cases. Running it is quite simple, the usage is shown below:

./body-ugid-dist.py 
usage: body-ugid-dist.py [-h] --file FILENAME [--meta META]

This script parses an fls bodyfile and returns the uid or gid distribution on
a per directory basis.

optional arguments:
  -h, --help       show this help message and exit
  --file FILENAME  An fls bodyfile, see The Sleuth Kit.
  --meta META      --meta can be "uid" or "gid." Default is "uid"

I wrote about this previously for the SANS Digital Forensics BlogIf this kind of analysis interests you, join me for SANS 508: Advanced Computer Forensic Analysis & Incident Response in Phoenix in February of 2012.

Fourth Amendment Hard Disk Wipe

2011-11-22T10:21:00.001-06:00

Recently I replied to a thread on a mailing list about wiping hard disk drives.


source: http://cheezburger.com/tehpeanutbutterkitteh/lolz/View/2735751680

I'd just spent a few hours over a recent weekend playing around with the hdparm command in Linux because it has the ability to use the ATA Secure Erase feature, which is much faster and more comprehensive than software wipe utilities like the trusty Darik's Boot and Nuke. For example, I recently wiped a 500GB drive in just over two hours.

I was experimenting with hdparm and secure erase because I wanted to try it out and because I was prepping an old drive to give to a friend. After the secure erase finished and I verified that the drive contained no data, I wrote a little shell script to overwrite the entire thing with the text of the 4th Amendment of the U.S. Constitution. Something I was inspired to do after reading about @ioerror's overwriting usb sticks with the Bill of Rights.

I mentioned this script on a mailing list and a friend replied that I was "so subversive." Now, I'm almost certain the reply was in jest and that he doesn't honestly feel that way, but I suspect there are folks who do think it's subversive. I think it's a sad commentary on the state of the U.S. collective psyche when we consider Constitutional guarantees as subversive.

A handful of people replied to me that they wanted the script. Well, it's not pretty, nor fast and Hal Pomeranz and a thousand other Unix beards could probably come up with a better solution, but it works. I've added a measure of protection to it because I imagine some people will screw themselves with this, so be careful, mind your devices.

#!/bin/bash
# This is a hack I wrote to overwrite $1 with the 4th Amendment.
# It's not pretty, it's not fast, but it works.
# If $1 is a device, when it's full, errors will be thrown and not handled.
# If $1 is not a device, the block device that it resides on will eventually
# fill up, if this script is left running.

# The next line will cause the script to exit on any errors, like
# when the device is full. Hey, I said it was a hack.
set -e

echo "This hack overwrites $1 with the text of the 4th Amendment."
echo "ALL DATA WILL BE LOST."

echo "Are you absofrigginlutely sure you want to continue?"
select yn in "Yes" "No"; do
    case $yn in
        Yes ) exec > $1
            while : 
                do echo "The right of the people to be secure in their persons, " \
                "houses, papers, and effects, against unreasonable searches and " \
                "seizures, shall not be violated, and no Warrants shall issue, " \
                "but upon probable cause, supported by Oath or affirmation, and " \
                "particularly describing the place to be searched, and the " \
                "persons or things to be seized.";
                done;;
        No ) exit;;
    esac
done

To use this save it as a shell script on a Linux system and invoke it from the command line as <command name> <device name>. When the device is full, the program will exit on error. Enjoy.

Egress Filtering

2011-10-23T12:32:00.000-05:00

“It is not what enters into the mouth that defiles the man, but what proceeds out of the mouth, this defiles the man.”
-- Jesus

White Hat Security's Jeremiah Grossman recently tweeted the following quotes from info sec legend Dan Geer:

Geer is a genius, there can be no doubt. However, when I read this, it bothered me. I have worked in large enterprises where knowing everything was nearly impossible and yet default-deny egress filtering was in place and effective at limiting loss.

Certainly implementing a default-deny egress filter without careful planning will be a resume generating event, but not implementing it due to incomplete knowledge may have the same result.

And as I said in response to Jeremiah's tweets on Twitter, implementing a default deny quickly leads to knowledge, but again, you're going to want to do this in a well-communicated and coordinated way, with careful planning throughout the organisation and management chain.

Fuzzy Hashing and E-Discovery

2011-08-19T16:20:00.002-05:00

Recent work has made me consider an interesting role fuzzy hashes could play in E-Discovery.

In the last year I've worked a few intellectual property theft cases where Company A has sued Company B claiming Company B stole IP from Company A in the form of documents, design drawings, spreadsheets, contracts, etc.

In these cases Company A has requested that Company B turn over all documents that may pertain to Company A or Company A's work product, etc. with specific search terms provided and so on.

Company B argues they can't comply with Company A's request because they have documents relating to Company A and Company A's work product as a result of market research for the purposes of strategic planning and that turning over all of those documents would damage Company B.

In such cases, if Company A is concerned that Company B has stolen specific documents, maybe a better approach would be to request that Company B run ssdeep or another fuzzy hashing tool against all of their documents and turn over the fuzzy hashes.

Company A can then review the fuzzy hash results from Company B without knowing anything about the documents those hashes came from. They can compare the set of hashes provided by Company B against the set of fuzzy hashes generated from their own documents and make an argument to the judge to compel Company B to turn over those documents that match beyond a certain threshold.

24:DZL3MxMsqTzquAxQ+BP/te7hMHg9iGCTMyzGVmZWImQjXIvTvT/X7FJf8XLVw:J3oy+x/te7qmNmlYvX/xp8W

Facebook Artifact Parser

2011-08-14T02:18:00.000-05:00

If you have a Facebook account, take a look under the hood some time by viewing the source in your browser while you're logged in. Imagine having to deal with all of that for a digital forensics investigation. It's mind numbing, especially if all you want is who said what and when. I spent the better part of today brushing up on Python's regular expression implementation and put together this Facebook Artifact Parser that does a decent job of parsing through Facebook artifacts found on disk (as of the time of this writing).

In my case, I made use of this by first recovering several MB worth of Facebook artifacts from disk and I combined all of these elements into one file. Having done that, run this script from the command line giving the name of the file as the only argument. It works on multiple files as well.

Yahoo! Messenger Decoder Updated

2011-08-07T12:03:00.000-05:00

I'm working yet another case that involves Yahoo! Messenger Archives. I tried using JAD Software's excellent Internet Evidence Finder for this and it worked pretty well, but in the interest of double-checking my tools, I brushed off my old yahoo_msg_decoder.py script that I'd written a few years ago. It used to be interactive, meaning it was run with no arguments and would prompt for a username and a filename to parse, this was less than ideal for parsing a large number of files.

I have remedied that situation. The script now takes three arguments, one optional. The first is the username for the archive. Yahoo! Messenger Archives are xor'd with the username. The second argument is the name of the other party to the conversation and the third argument is the name of the dat file to process.

The nice thing about this is that you can now create a for loop like the following from a Linux environment and parse multiple files at once:

for i in $(ls *.dat); do echo; echo "== Parsing $i =="; yahoo_msg_decoder.py --username=joebob --other_party=billybob --file=$i; echo "== Finished parsing $i =="; echo; done

The output of this for loop can be redirected to a file.

My script is still not perfect. On some dat files it doesn't properly xor the data and yields garbage. I have not determined why that is the case yet.

As for IEF, I'm not sure why, but running it over the same dat files as my script, it dropped some portions of the conversation. I will be reporting the issue to JAD. But it's yet another reminder of the importance of testing your tools and confirming results.

update: After posting this, I remembered that Jeff Bryner had written a utility for this and it is still vastly superior to my own. I just verified that the link I have to his yim2text still works. Check it out.

Awk regtime bodyfile adjustment

2011-05-30T14:20:00.000-05:00

Here's an awk one liner for adjusting regtime bodyfile time stamps, in this case we're adding 600 seconds:

awk -F'|' 'BEGIN {OFS="|"} {$9=$9+600;print}'

One thing to consider when adjusting time stamps to compensate for clock drift, clocks don't drift all at once, but over days, weeks and months. Adjusting time skews affects everything all at once.

Time again

2011-05-18T23:15:00.000-05:00

I gave a version of the Time Line Analysis talk at Cyber Guardian earlier this week. Some in the room asked if the slides would be made available. As promised, here is a link to the deck. Enjoy.

MapReduce for simpletons

2011-04-26T18:32:00.000-05:00

Data reduction redux and map-reduce is the title of my latest post at the SANS Digital Forensics Blog. I mentioned in my previous post there on using least frequency of occurrence in string searching that there would be a follow up.

The point of the new post is to sing the praises of @strcpy over on the Twitters. He helped me out by writing a short shell script that is, in essence, map-reduce for simpletons like me. I am constantly amazed by some of the members of the info sec community who will take time to help out near total strangers.

strcpy's script wasn't just helpful, it was educational. I'd read about map-reduce before, but it never really clicked until I saw strcpy's script. The scales have fallen from my eyes and I'm now adapting his script for other kinds of tasks.

Check out the post and if you find it beneficial and you ever get to meet strcpy in person, buy him a drink or a meal and tell him thanks, I plan to do the same one day.

Scalpel and Foremost

2011-04-25T22:24:00.000-05:00

The crew over at Digital Forensics Solutions announced the release of a new version of Scalpel with some exciting new features. Check out their post for the full details, but here are three I was most interested in:

Parallel architecture to take full advantage of multicore processors

Beta support for NVIDIA CUDA-based GPU acceleration of header / footer searches

An asynchronous IO architecture for significantly faster IO throughput

Digital forensics is time consuming so any speed gains we can make are welcome ones.

Over the last few days, I've had a chance to play with the new version of scalpel on my 64-bit Ubuntu system with 7GB of RAM. I downloaded the source and followed the directions in the readme to configure and compile the binary.

I then ran some carves against a 103GB disk image from a recent case. The command line I used was:

scalpel -b -c /etc/scalpel.conf -o scalpel-out/ -q 4096 sda1.dd

The -q option is similar to foremost's -q option in that it tells scalpel to only scan the start of each cluster boundary for header values that match those specified in the config file. In my test, I used the two doc file signatures in the supplied example scalpel config file. The nice thing about -q is that you can provide the cluster size. With foremost -q will scan the start of each sector by default, you'll have to also add -b to get similar functionality out of foremost.

I ran scalpel with the Linux time command so I could determine how long the command took to complete. Scalpel carved 6464 items that had byte signatures matching those in the configuration file. According to the time command, this took 52 minutes and 40 seconds.

Manually verifying that all 6464 files are Word docs would be time consuming. In lieu of that, I followed Andrew Case's suggestion and used the following command from within the scalpel-out directory:

for i in $(find . | grep doc$); do file $i; done | grep -i corrupt | wc -l

The result was that 2707 of the 6464 files were found to be "corrupt" according to the file command. This is not an exact measure of the accuracy of scalpel's work, but it gives us a ballpark figure. If my math is correct that's a false positive rate of 41%. Just remember, these are rough figures, not exactly scientific.

Next I configured foremost to use the exact same configuration file options and similar command line arguments (recall I had to use -b with foremost) and ran the carve against the same image. The command line I used was:

foremost -c /etc/foremost.conf -i sda1.dd -o foremost-out -q -b 4096

Again, I used the time command to measure how long this took, 47 minutes and 32 seconds later, foremost finished having carved 6464 files. I used the same measure for accuracy as with scalpel, running the following command from within the foremost-out directory:

for i in $(find . | grep doc$); do file $i; done | grep -i corrupt | wc -l

The result was that 2743 files came back as "corrupt" according to the file command. Interesting. Both tools used the exact same signatures, both carved exactly the same number of files, yet foremost was approximately 1% less accurate, though at 47 minutes compared to scalpel's 52 minutes, it was almost 10% faster.

Conclusion:
It's hard to draw conclusions from one simple test. I think it's great that scalpel is under active development and for those who can take advantage of the CUDA support, it could be a huge win in terms of time and time is against us these days in the digital forensics world.

The other big plus, is that it's great to have another tool that we can use to test the results of another tool. I will continue to experiment with scalpel and look forward to future developments and I thank the developers of both tools for their contributions to the community.

Forensic string searching

2011-04-23T09:30:00.000-05:00

Can the principle of "least frequent occurrence" be applied to digital forensic string searches?

Late last night (or painfully early this morning) I published a new post over at the SANS Digital Forensics Blog. The post is called "Least frequently occurring strings?" and attempts to shed some light on that question.

I've used this approach on a couple of recent cases, one real and one from The Honeynet Project's forensic challenge image found here, this is the image the post contains data from.

I really knew nothing about the Honeynet challenge case, but in less than half an hour, I'd located an IRC bot using the LFO approach to analyzing strings. Of course the Honeynet case is quite small, so the technique worked well, on larger cases from the real world, I expect it's going to take longer or maybe not work at all. Nevertheless, LFO is a concept that other practitioners have been applying for some time now.

There's lots of other goodies in the post, like moving beyond just using strings to extract ASCII and Unicode text from disk images. If you have a decent system and a good dictionary file, you can reduce this set of data even further to lines that actually contain English words.

Check it out, I hope the world finds it useful.

Incident Response Triage

2011-03-16T10:02:00.000-05:00

Your phone rings, it's the Help Desk. They are calling you because they've got a few dozen systems that have been hit with malware that apparently came into the organization via phishing. Unfortunately, your team isn't large enough to respond to all of these systems simultaneously. You've got to quickly prioritize.

You call the members of your team together and start delegating tasks. One person contacts the email admins and finds out who received the phishing email and compares that list against the one the Help Desk gave you. The email admins remove the offending message from user mailboxes and blacklist the sender. You ask them to send a copy of the message to you so you can dissect it and begin the process of analyzing the malware.

You learn of another dozen potentially compromised hosts from conversations with the email admins.. You add them to your list. How do you prioritize your response to these victim systems? Let's say your company is very large, Fortune 100, and has been through a series of mergers and acquisitions over the last several years, nearly all of the names on the list of affected users are unknown to you. On the one hand, this may be good as it's likely none of these individuals are C-level execs. On the other hand, you've now got to figure out who these people are and what data they have on their systems and what data they have access to and who their local IT support personnel are.

What are your next steps? Do you contact each user and survey them, asking what kind of data they deal with and have access to? Do you ask who their IT support person is? How accurate is the information you're going to get? What if some of these systems are multi-user and the user you're talking to is unaware of the special projects and associated data?

Aside from questioning users, what other information gathering do you need to do? Does your organization have good exfiltration monitoring and logging in place? Do you have the ability to pull those logs and see what, if any, data has left the org? Do you have the ability to rapidly block outbound connections to the malware's command and control networks?

I know I'm asking more questions than I'm answering, partly this is stream-of-consciousness writing, but I'm also soliciting input on IR triage for a project I'm working on. I've started a little IR triage tool I'm calling Windows Automated Incident Triage or WAIT. Here is the current capability roadmap for WAIT: identify users on a system and their privilege levels, catalog the data those users have recently accessed on their systems, create a list of file shares those users have recently accessed, gather available web history, collect information about the system's OS revision and a list of installed software.

My hope is that this information will be useful to IR professionals in a situation like that above. I want a tool that can be used to help prioritize IR. What artifacts am I missing that may also be useful?

And of course the tool will be open source, likely released under a BSD style license.

How to find base64 encoded evidence

2011-01-09T10:45:00.000-06:00

Today I released a post over at the SANS Digital Forensics Blog discussing how to find evidence that may have been base64 encoded and therefore not found by traditional tools that categorize files based on magic numbers.

The technique is really simple, but I hadn't seen it discussed elsewhere, perhaps because it's so obvious.

Enjoy.

Update: Here's a text file containing some magic byte sequences for common image types that have been base64 encoded: http://trustedsignal.com/forensics/b64_enc_img_types.txt.

Trust those time stamps?

2010-11-02T15:19:00.000-05:00

I've got a new blog post up at the SANS Digital Forensics Blog titled Digital Forensics: Detecting time stamp manipulation. The post is my effort to demonstrate that time stamp manipulation on systems running NTFS can be spotted (for now) if examiners take the time to fully investigate all of the available evidence (i.e. compare $STANDARD_INFO and $FILE_NAME) time stamps.

This is another barrage in my quest to get Brian Carrier, a true giant in this field, to add the capability to fls to pull $FILE_NAME time stamps into the body file format so we can build time lines using mactime that include both $STANDARD_INFO time stamps and $FILE_NAME time stamps.

Fortunately, Mark McKinnon has written a tool called mft_parser that will do this. As soon as that tool is available for wider release, I'll post a link.

Oh and I said for now, because I'm confident that the right rootkit will be able to manipulate $FILE_NAME time stamps as well as $STANDARD_INFO time stamps. In such cases, we'll have to rely on time stamps in other artifacts.

Windows Persistence

2010-10-22T23:14:00.000-05:00

I wrote up a post on the SANS Digital Forensics blog titled Digital Forensics: Persistence Registry keys where I gave a couple of links to text files containing Registry keys for Windows XP SP3 system that I'd run Autoruns on to gather a list of Registry keys that could (possibly) be used as persistence vector's for malware.

I have collected similar lists for Windows Vista and Windows 7. The files are available at trustedsignal.com/IR.

I see what you did there

2010-09-17T15:13:00.000-05:00

I had the pleasure of presenting at Security BSides in KC this morning. Shouts out to hevnsnt and bsideskc for putting on the event.

Unfortunately, my schedule didn't allow me to see all of the talks, but what I did see was valuable, though the "face time" with my peers in the field (I met hal_pomeranz and kriggins in the flesh) was probably more fun than presenting or watching any of the presentations I was able to see.

My talk was called I see what you did there, and was about time lines in forensic investigations and incident response. Some of the material in the talk comes out of SANS 508: Computer Forensic Investigations and Incident Response, a course I've had the pleasure of teaching a few times. Thank you to Rob Lee and his contributions to the field over at the SANS Digital Forensics Blog. Obviously, the six day course is able to cover this topic in much more detail than I was able to do in one hour.

BSides is awesome. Everyone should submit talks, it makes you better, even if you can't talk about an original tool or concept, many people don't know what you know and when you prepare to share it with them, you become more knowledgeable than you were when you started.

Givem 'em enough rope

2010-07-10T10:33:00.000-05:00

I mentioned on Twitter that having worked in relatively unrestricted environments (higher ed) and in highly restricted environments (banking) and in between, that in my experience, those environments with more draconian policies have better security.

No sooner than I hit "Send", I realized I should have also said "controls" because policies by themselves are pretty lousy security controls.

Since Twitter is less than ideal for elaborating on, well, much of anything, let me explain what I mean, for those who may not agree.

Most of my work in information security has been in the incident response and forensics space, with a few years in application security -- I'm a recovering developer.

During my time in higher ed, most users ran as admin and could install whatever they wanted, whenever they wanted. They could browse to any web site of their choosing. This is the "give 'em enough rope" approach to information security. The problem is that this doesn't just lead to the users hanging themselves. Shops operating in this manner are giving their employees enough rope to hang the entire organization.

During my tenure in the financial sector, few people ran as admin, application whitelisting was in effect for most people and web content filtering kept most people from browsing to known-malicious or "inappropriate" domains.

The results of these two disparate approaches was striking. Higher ed was an incident responder/forensic investigator's dream job as there was never a shortage of interesting work. By contrast, the bank didn't have any full time incident response and forensics folks. During my two years at the bank, we had less than a handful of issues and they were all drive-by-downloads from rogue advertisements on mainstream web sites.

I believe most organizations could greatly improve their security and reduce costs by taking away internet access for those employees that don't need it and greatly restricting internet access for those who do need it. It's unpopular, it's draconian, but it works.

Don't let your users run as admin. I can't believe we're still seeing this as much as we are. If you have some users who need admin access, give them separate accounts to use when they need that level of access.

Whitelisting. It sucks. It's a horrible pain for the users and those who have to maintain it. Before I worked in an environment that used it, I dismissed it completely. But as much as it sucks and is painful to implement and maintain, it will reduce the number of security incidents that you have to deal with. Note, if you take away your user's admin rights, you may not need whitelisting.

I've said almost nothing of application security, but this is another area where more restriction leads to greater security. Limit your developers access to production environments, don't let them adapt new technologies/frameworks/libraries without first taking the time to review the security of those technologies. Don't let devs move forward on projects until threat models have been developed and threats have been addressed. Don't let code go to production without some type of review, don't push applications to prod without security testing those apps, etc.

Yes, this is expensive and time consuming, but in my opinion it's a pay now or pay more later scenario. Spending thousands up front may save you from spending hundreds of thousands after a breach.

Will all of this save every organisation 100% of the time? No, but it will significantly reduce the number of incidents. Will it be popular with employees? No, but watching Double Rainbow Song is probably something they should do on their own time and on their own computer.

Security will never be perfect, but a big part of the reason it is as broken today as it is, is because we haven't made the unpopular decisions that need to be made.

Wifi Security Slides

2010-06-29T13:54:00.000-05:00

I had the privilege of being invited to speak about wireless security to the U.S. Army's Combined Arms Center, Office of the Chief Information Officer & G6's Information Security Symposium at Fort Leavenworth in Leavenworth, Ks on June 29, 2010. Yes, that is a "challenge coin" that says "Presented by the commanding general..." in the photo, a nice addition to the collection.

Judging by the comments I received after the talk, it went well. I didn't drop any "1337" hax0rs or any zero day, in fact, in the few weeks I spent preparing for the presentation, it seemed to me that there isn't too much new stuff coming out for 802.11, or more likely, I don't travel in the right circles.

The talk drew entirely on the research of others and I tried to give credit wherever it was due. Thank you Josh Wright for letting me stand on your shoulders. I did tell people to visit your site and take your course.

I have made the slides available in PDF and PPT format. There are a few canned video demos in the PPT version that are obviously not in the PDF version and the PPT version contains copious notes, not found in the PDF.

Thanks again to Austin Pearson, Major Fraley, et al for the opportunity.

PDF Slides

PPT Slides

Career change management

2010-04-14T17:40:00.000-05:00

Twenty-three months ago I returned to the career field I am drawn to more than any other after an equally long hiatus. I was happy to be back in an information security role, even if I did give up a "director" level position for more of an in-the-weeds role. I do like weeds.

For the last two years, I spent about 99% of my working life on web application security. Week one at my new job had me in training, learning how to use Fortify's static code analysis tool. I'd done manual code review before in both developer and security roles. In a previous position, I'd evaluated the open source static analysis tools and found them to be less effective than performing data flow analysis manually. Granted, data flow analysis may not catch all security flaws, but it provides good coverage against attacks from user input.

After two years working with Fortify, I'm happy to be done with it. Code review is difficult for humans to do well. Humans can barely write good software. Creating an expert system that does code review effectively is a "hard problem."

To be fair, the company I worked for was using Fortify out of the box with no custom rules and a decision was made early on to review all findings regardless of confidence level and severity rating. Decisions have inertia. I was told by someone within Fortify that our usage was "really aggressive" and learned that many similar enterprises were only reviewing issues with high confidence and high severity. Perhaps our aggressive program skewed my perception of the tool's ability to find vulnerabilities.

I will say this in Fortify's favor though, requiring developers to use the tool and to audit the issues in their code (audit does not always mean fix), does educate developers who take the time to read and understand the issues. So even if the tool is suboptimal for finding real security issues, it will make at least some of your developers think and learn about security issues and in the end having developers who understand security issues and who can write safe code is probably more valuable than having a tool that can reliably find flaws.

I've glossed over it thus far, but for those reading between the lines, yes, I've moved on from the day job that I've had for the last two years. A few of you knew I worked for a regional bank holding company doing application security work (mostly web related). It was the most demanding position I've ever held, note demanding does not equate to challenging, though at times the position was both. All-in-all my experience over the last two years was very positive. I worked with a team of smart folks and pushed myself to learn some critical skills.

I'm working in a new role now that is more aligned with my interests in incident response and forensics, though I will continue to work in the web application penetration space as often as I can. But if you ask me to do code review, I'll likely be doing it the old fashioned way, tracing data flows, line-by-line.

With my professional and personal interests more in sync than they have been for several years, I hope to be able to post some new research here or at the SANS Digital Forensics Blog, which I've been helping Rob Lee manage since its inception.

Musings on recent high profile hacks

2010-01-13T22:01:00.000-06:00

So Google got hacked. You can read about it all over the place. The details are few, but from the sounds of the articles I've read, Google has been hit by what Mandiant likes to call the Advanced Persistent Threat or APT. In a nutshell, APT is likely nation-state backed hackers. Note that we don't have any idea which nation-state.

Google says they lost intellectual property, but claims that no customer data was compromised. Ok. I have actually worked more than one incident response case over the last few years where I felt we could honestly say that and that was after days and days of reviewing logs and running leads to ground. Maybe Google is being forthright about that. Maybe they are saying it for CYA. I don't think it's all that interesting.

What does interest me are the non-obvious ways that attacking Google can be leveraged into devastating attacks. Own Google, own the net.

Adobe has also been hacked. I think it would be a sad irony if they were hacked via PDF malware sent to an executive in the company. I also think that's highly likely.

And the US military has admitted that the unmanned drones they've been using in theaters of operation around the world are having their transmissions sniffed by $26 software readily available on the net. And the government has known that the transmissions were unencrypted and could be intercepted for like five years.

People have been crying about this quite a bit and about how shameful it is, etc. What I haven't heard any one else talk about is how the US government could possibly use this vulnerability to their advantage. First, they could reverse engineer the $26 software and see if it has any remotely exploitable vulnerabilities and use those to attack those intercepting the traffic.

A more obvious attack would be to feed bogus images through the drone to those sniffing the traffic, thus launching a misinformation campaign.

There are many facets to compromise.

2009 SANS Forensics Summit Recap: Day Two

2009-07-16T19:44:00.000-05:00

In my previous post I recapped day one of the 2009 SANS Forensics Summit. In this post, I'll continue with coverage of day two, but first, I have to say that I did cut out for a few hours during day two to have lunch with my friend mubix from Room362.com so I apologize in advance for not being able to comment on things I didn't see.

Ovie Carroll, Director of the Cybercrime Lab at U.S. Department of Justice Computer Crime and Intellectual Property Section started off day two. Carroll is co-host of the Cyberspeak podcast and like Richard Bejtlich, Carroll gives a great presentation accompanied by an entertaining slide deck. One of the things I really liked about Carroll's presentation was that he took time to update it with information that had been presented the previous day. There weren't a bunch of updates, but it was nice to see that he thought content from the previous day was as valuable as I did and that he took the time to make the updates at all demonstrated how much he cared about the subject matter.

Carroll spoke about trends in and the future of forensics from a law enforcement perspective. One of the key take aways from Carroll's talk was that there is a mountain of work facing law enforcement and they are having difficulty keeping up. He mentioned that it was not uncommon for some agencies to have systems in their possession for 18 months before they get a look at them. Having worked for defense attorneys (prosecutors never call me) for a number of years now, I haven't seen delays quite that long, but I don't doubt it for some agencies.

Clearly there are a number of factors contributing to the delay. One is that law enforcement is interested in analyzing computers even in traditional crimes because they have found so much good evidence on people's hard drives. Two, there simply aren't enough people doing this work due to lack of qualified personnel and due to budget constraints the problem likely won't go away, ever. Lastly and no less importantly, there's just a ton of information being produced each year in this digital age. Carroll said that in 2008 more content was produced online than humanity produced in traditional forms (paper and ink) over the last 5000 years. Sure, not all of that data is relevant to case work, but some of it is and it takes time to analyze what's relevant.

Carroll has been advocating for a phased approach for a while now and he repeated the call during his talk. Law enforcement agencies should take a triage approach and try to build enough of a case without completely analyzing systems that they can get suspects to plea bargain and thus clear out some of the case load, at least for the more mild offenders. This is something I've told students as well, yes we want to analyze every piece of evidence that we collect, unless of course we can build a strong case without doing all that comprehensive work and short-circuit the process through a plea bargain.

One more thing about Carroll, he's funny. You want that in morning speaker.

Following Carrol, Chris Kelly Managing Attorney for the Cybercrime Division of Massachusetts' Attorney General's Office addressed the audience. Kelly had some great stories about some really stupid criminals, the ones who get caught, generally are, but one guy rose above the rest by snapping a picture of himself with someone's cell phone while he was in the act of robbing that someone's home. Good times.

Kelly started out talking about how much things have changed in the cybercrime world. We've gone from phone phreakers, defacements and obnoxious worms to organized criminal networks, terrorism and traditional crimes that involve computers as sources of evidence. As an example of the latter, consider a case in my area where a college professor was convicted of killing his ex-wife. One piece of evidence found on his home computer was search history about ways to kill people. He claimed he was doing research for a novel. Along these lines, Kelly brought up the case of Neil Entwistle who killed his wife and daughter. In his search history were queries about how to kill people.

Kelly also spoke about some of the training they are offering to law enforcement including the need for first responders to stop pulling the plug and to perform collection of volatile evidence. He played a hilarious clip from CSI of cell phone forensic analysis that had everyone in the room laughing.

At this point, I'm sorry to say, I had to cut out, but a user panel assembled to discuss aspects of forensics in law enforcement. The panel was to have included Carroll, Kelly, Andrew Bonillo, Special Agent/Computer Forensic Examiner at the U.S. Secret Service; Richard Brittson, retired detective, New York City Police Department; Jennifer Kolde, Computer Scientist with the FBI San Diego Division's National Security Cyber Squad; Cindy Murphy, detective, City of Madison, WI Police Department; Ken Privette, Special Agent in Charge of Digital Evidence Services, United States Postal Service Office of Inspector General; Paul J. Vitchock, Special Agent, Federal Bureau of Investigation, Washington Field Office; and Elizabeth Whitney, Forensic Computer Examiner, City-County Bureau of Identification, Raleigh, NC.

I apologize if I missed anyone on the list, because I missed the panel, I'm going off of the agenda so some of these folks may not have been there and others may have been on the panel in their place. I have looked over some of the presentations that were given and I'm sure I missed some great content and as someone who frequently works opposite law enforcement, I wish I could have caught this panel.

After lunch, Dr. Doug White Director of the FANS Lab at Roger Williams University spoke about several different topics related to forensics and the courtroom including some cases where admissability of evidence came into play. I got a little lost at one point while White was speaking about this. His slides referred to US. V. Richardson 583 F. Supp. 2d 694 (W.D. PA 2008) with the sub-bullet referring to hacker defense, but the only thing I can find about the case online indicates that there were scoping issues with a warrant rather than a hacker defense.

White brought up another interesting case, U. S. v. Carter 549 F. Supp. 2d (D. Nev. 2008), discussed here, where the IP address of a suspect's system was deemed circumstantial evidence and could not be used to tie an individual to the crime. Lesson for investigators, get as much supporting evidence as you can.

White talked about the Adam Walsh Act that limits defense attorneys and experts to "reasonable access" to the evidence in cases that involve the exploitation of children. Reasonable access generally means at the law enforcement agency during normal business hours. This is a well intentioned law that has cost me some business but if it prevents children from suffering at the hands of incompetent practitioners who lose hard drives or otherwise leak evidence, then it's a good thing.

One great recommendation White made was to spend downtime coming up with simple ways to explain complex topics. There are lots of things those of us in tech take for granted, like IP addressing and NAT, but when we have to explain them to non-technical folks it can be difficult. Spending time to write clear and easy to understand explanations that can be quickly added to the appendix of a report saves time. It's like developers reusing code.

Following White's talk Craig Ball, trial lawyer and forensic expert, touched on this same idea during his lightning talk as part of the user panel on challenges in the court room. Ball is a wonderful presenter. He struck me as a very intelligent, thoughtful and friendly gentleman (he's a lawyer?!). Ball had a great slide deck loaded with graphics including some simple animations that he uses to explain complex topics in simple ways to members of the jury, things like how a hard drive works.

Ball also mentioned using visualization software for turning timelines into nice looking charts. I believe he said he uses a product called Time Map, but a quick search reveals there are quite a few different products on the market. Check out Ball's website where he has loads of materials available for free. I would love to see Ball at work in the court room. I hope to catch him giving a longer presentation at some point in the future.

Also on the panel with Ball were White, Gary Kessler, Associate Professor of Computer and Digital Forensics and Director of the Center for Digital Investigations at Chamberlain College; Bret Padres Direcor of Digital Forensics for Stroz Friedberg and co-host of the Cyberspeak podcast and Larry Daniel, principal examiner for Guardian Digital Forensics. I may have missed someone, Dave Kleiman was on the agenda, but his slides aren't on the conference CD and I can't remember him being on the panel, this is not to say that if he was on the panel, he didn't have anything noteworthy to say. Rather, it's a reflection on my own poor memory and the fact that I'm writing this more than a week after the event.

Kessler's question was to rank the qualities in order of importance that an investigator should have and to explain his ranking. I liked Kessler's answer because he took the list given to him (analysis skills, acquisition skills, data recovery skills, report writing skills (Kessler expanded this to communication skills), law enforcement background, computer science background, problem solving skills, integrity and caution) and he added his own qualities of curiosity, technical astuteness and tenacity.

Kessler's overall number one choice was integrity, something I happen to agree with. And his least important qualities were a computer science background followed by a law enforcement background. For those of us in the field who lack a computer science degree and a law enforcement background, it's easy to agree with Kessler's putting those at the bottom of the list. His second most important quality was technical astuteness. Oddly enough, I know some folks with computer science degrees, who have difficulty with technology outside of their narrow field of specialization.

Kessler made a point about good examiners that I've heard repeated by others in the field. So much of the job is about being tenacious. Hard problems are hard and many times there are no quick wins, the examiner who sticks with it and works through the adversity is the one you want working for you.

At this point, I had to cut out and catch a flight. All in all, this was the greatest incident response and forensics focused conference I've attended. If you work in the field, you should try and attend next year, this is not a normal SANS event, it's really a single track conference bookended by the training that SANS if known for.

I hope to see you there next year.

2009 SANS Forensics Summit Recap: Day One

2009-07-14T22:57:00.001-05:00

I had the great pleasure of attending and participating as a panelist in the 2009 SANS What Works Summit in Forensics and Incident Response. I covered my presentation in a previous post, but wanted to share my thoughts on some other aspects of this great event.

The Summit was a two day event, with day one focusing mostly on the more technical aspects of forensics and incident response. Day two's focus was more on the legal side of things, though Eoghan Casey of cmdlabs did give an excellent technical talk on mobile device forensics on day two.

Day one kicked off with a keynote address by Richard Bejtlich. This is the second year in a row Bejtlich has addressed the attendees and from what I gather, his talk this year was sort of a continuation of his talk from last year. If you read Bejtlich's blog, you know he's a critical thinker and has made some valuable contributions to the field. He knows how to put together an engaging talk and a good slide deck to go with it.

I did have a slight "uh oh" moment during Bejtlich's address when he mentioned his concept of creating a National Digital Security Board. I have been reading Bejtlich's blog for years, but apparently missed that entry entirely. The "uh oh" was because that was pretty closely related to the theme of my panel presentation. In a nutshell, my take was that incident responders ought to be much more open about what we're dealing with in the same way that the National Transportation Safety Board publishes over 2000 reports each year regarding transportation failures.

Following Bejtlich was an excellent talk by Kris Harms called "Evil or Not? Rapid Confirmation of Compromised Hosts Via Live Incident Response". I appreciated Harms' talk because it was basically a talk from the trenches, very nuts and bolts, covering a list of different techniques and tools that incident responders can use to quickly assess a potentially compromised system.

Harms covered many of the tools commonly used by incident responders, but I picked up a few new tactics. One of them was his use of Sysinternals Autoruns and it's capacity to sort out signed and unsigned code. Certainly a criminal could go to the trouble to create signed code and many companies produce legit code that's unsigned, but one of the things incident responders, like forensic examiners need to do is quickly reduce the size of the data to parse and this is one possible technique.

Harms also spoke about the Advanced Persistent Threat something we should all be giving more attention. APT's frequently make use of rootkit technologies to hide themselves. Harms gave some examples of using Handle, again a Sysinternals tool that most IR folks have used, but it was interesting to note that Handle could be used to work backwards from open files to process IDs and that usually rootkits aren't able to hide themselves from this backwards approach.

Harms talk alone would benefit any incident responder, but the Summit was just getting started. After Harms, a panel of incident responders took the stage, including Harlan Carvey, Harms, Chris Pogue and Ken Bradley and myself. Each member of the panel gave a lightning style talk answering a question of their choosing. The general consensus of the group was that the best tool for incident responders is still the gray matter between one's ears. Following the panelist's presentations, members of the audience had a chance to ask questions. This format was followed for all the panels during the Summit. It's a great opportunity for practitioners to pick the brains of leading experts.

Following lunch, Carvey took the stage again and talked registry forensics. When it comes to the Windows registry, Carvey has done more for the Windows IR and forensics community than any other individual. If you are an incident responder or digital investigator and haven't picked up a copy of his book you really should purchase a copy or watch the SANS Forensics Blog where we'll be giving away a few copies courtesy of Syngress Publishing. Carvey has written some great tools for pulling useful information from the registry and has made them freely available from his web site. One thing he said during his talk and that is repeated in his book is that the Windows registry is a log file. Given the fact that keys have last write time stamps, this is true and can be very useful for making a case. Carvey's a great speaker, if you have a chance to see him talk, don't pass it up.

Following Carvey was another panel discussion on essential forensics tools and techniques. Jesse Kornblum spoke about dealing with foreign languages in malware. Kornblum has an amazing mind and has made many great contributions to the field. Hearing him speak was another among the many highlights of the Summit. Troy Larson answered the question, "What forensic tool needs to be created that doesn't exist yet?" His answer was "a tool to perform intelligent network imaging of volume shadow copies." If you don't know, volume shadow copies are bit-level diffs of all the clusters on your Windows Vista and later volumes. Obviously, there's a wealth of useful data in there, but as of yet, getting at the data is a labor intensive process and sadly many practitioners don't even bother.

Also on the panel was Mark McKinnon of RedWolf Computer Forensics, author of numerous forensics tools including Skype Log Parser, CSC Parser (for offline files) as well as a number of parsers for a variety of browsers. McKinnon answered the question "What are 2-3 major challenges that investigators now face or will face in the near future?" His answer was the astounding amount of new software and hardware that is flooding the market including the latest smart phones, gaming consoles, Google Wave, etc.

Jess Garcia from One eSecurity spoke about using different tools and different approaches depending on the type of case being worked. On one of his slides he mentioned cases involving cloud providers. I can just imagine the headaches that's going to present in the future.

At the end of the panel, Rob Lee asked the panelists what their favorite forensics tools was or what they used most often and I believe everyone of them said X-Ways Forensics and WinHex.

Next, Jamie Butler and Peter Silberman from Mandiant spoke about memory forensics and ran through some demos. On the day of their talk, they also released new versions of Memoryze and Audit Viewer. These two are whip smart and it was great to see their work in action.

The writing has been on the wall for a few years now that collecting memory dumps could replace a bunch of more traditional live response steps and with the advances that these tools bring, there should no longer be any doubt that collecting memory should be the first step in any incident response. There are bits of information you can get from memory that you can't get from any other tools. One of these is time stamps for socket connections. To say nothing of memory resident malware. Memory analysis is the future and the future is here now (though it may not be evenly distributed, as has been said).

Even if you're dealing with a system that doesn't currently have good analysis tools available for its memory dumps, don't underestimate the ability of geniuses like Butler and Silberman to create tools that may one day help your case and in the meantime, there's still scads of information you can glean from a simple strings search.

Following Butler and Silberman, Brendan Dolan-Gavitt a post-grad at Georgia Tech and a contributor to Volatility talked about and demoed some of his work parsing registry structures from memory dumps.

At that point, my brain was pretty full so I checked out for a bit and went to dinner, but made it back in time to catch the live recording of Cyberspeak. It was fun to watch the show and there was some great discussion between Ovie Carroll Larson and Craig Ball. I wish the members of the audience participating in the discussion could have been mic'd because there were lots of smart comments.

All in all, it was an amazing day. This was only my second time being in Washington D.C., my other visit being for Shmoocon and I considered cutting out to go do some sight-seeing, until I got there and realized there was going to be some world class content that no one in their right mind would want to miss.

I know it's the intention of the organizers to post as much of the presentations as possible, but as of this writing the files aren't available. Watch the SANS Forensics Blog for an announcement once the presentations are posted.

I'll post my day two recap in the next few days.

SANS Forensics Summit

2009-07-07T20:20:00.001-05:00

Rob Lee invited me to participate on the Incident Response panel at the SANS Forensics Summit. The panel consisted of some very well known and well respected experts in the field like Harlan Carvey, Kris Harms, Chris Pogue and Ken Bradley. Needless to say, it was a real privilege for me to be on the panel with these guys.

As panel members, we were each tasked with answering a question about incident response. Rob raised the questions, but gave us the option to answer a question of our own choosing based on the theme of the Summit. In a nutshell, the theme of the Summit was that given all the great advances that have been made in the incident response and forensics, what are the new essential techniques, tools and/or methods that incident handlers and forensic investigators should be using in their work.

Never one to take what's given to me without twisting it a bit, I took liberties with the theme in an effort to convey what I think is one of the powerful new ideas in information security based on Adam Shostack's and Andrew Stewart's book, The New School of Information Security.

My question then was basically this: Given that incident response has advanced greatly over the last decade, largely due to necessity because information security operations is pretty bad, as evidence consider T.J. Maxx's loss of 94 million credit card numbers in 2007 or T-Mobile's loss of 17 million records in 2008 or the untold millions of records lost by Heartland Payment Systems and of course the countless smaller failures each year that don't get much attention. Given all of that, what should incident handlers be doing to help improve information security operations overall?

I had five minutes to answer this question. That's not much time and some of my argument was lost due to time so I wanted to publish the slides for my talk here so folks could download them and take a look at the presenter notes and hopefully get a feel for where I was coming from.

As for my answer, as I mentioned in my talk, I strongly believe that information security is like that person who lost their keys on a darkened street and was searching for them when a stranger came by and offered to help. After several minutes of looking and finding nothing, the stranger asked, "Are you sure you lost them here?" And the person responded, actually, I lost them up the street, but the light is better here.

Too many information security operations teams are spending valuable cycles on the wrong things and it's not necessarily their fault. If you believe Shostack and Stewart, it's because we don't have adequate data to approach our tasks in a more scientific way. Info sec is currently being practiced as more of an art than a science and until we start gathering good metrics about failures, we may continue to focus on the wrong things.

So, I put the charge out for incident response firms to be more open about the failures they are seeing and to follow Verizon's lead and in fact, exceed it. We need more details about some of the most sophisticated and successful attacks. We need to know exactly how our defenses are failing. Data breach notification laws are well and good, but they generally give us very little insight into what went wrong.

And with that intro, here's the presentation. You'll want to open it in a new window and click the option to view the presenter notes on the Actions menu, otherwise it's mostly old photos:

How quickly we forget

2009-06-30T11:54:00.000-05:00

Date: Thu, 8 Jan 2009 08:26:45 -0800 (PST)
From: Rob Lee
Reply-To: Rob Lee
Subject: Re: [GCFA] Compiling evidence boils down to a matter of time
To: Dave Hull
Cc: GCFA
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="0-496094974-1231432005=:13648"
Message-ID: <1401.13648.qm@web42107.mail.mud.yahoo.com>

--0-496094974-1231432005=:13648
Content-Type: text/plain; charset=us-ascii

Done. That takes care of Windows 7 and Window Server 2008. Can you verify it can adjust all four timestamps or just a few of them? We can then add that to our list of known default programs. Also, can you document how it is used and what traces are left in its use?

What type of beer do you like and what is the next SANS conference you will be at?

--Rob

______________________________________________________________________________________________

________________________________
From: Dave Hull
To: Rob Lee
Cc: GCFA
Sent: Thursday, January 8, 2009 11:11:17 AM
Subject: Re: [GCFA] [HTCC] Compiling evidence boils down to a matter of time

Interesting thread. Windows 7 and Windows Server 2008 ship with
Powershell. Powershell can be used to modify timestamps. See this
entry on my blog for more info:

http://trustedsignal.blogspot.com/search/label/timestamps

Where's my six pack? ;)

--
Dave Hull
Trusted Signal
CISSP, GCFA, GCIH, GREM, SSP-MPA, CHFI
Public key: http://trustedsignal.com/pubkey.txt
Fingerprint: 4B2B F3AD A9C2 B4E1 CBDF B86F D360 D00F C18D C71B

"Great minds discuss ideas; Average minds discuss events; Small minds
discuss people." -- Eleanor Roosevelt

--0-496094974-1231432005=:13648

From New School of Information Security to Incident Response

2009-06-24T23:59:00.000-05:00

The SANS Forensics and Incident Response Summit is just around the corner. Judging by the agenda it's going to be the best event for forensics and IR professionals for 2009.

Of course, I'm biased. Rob Lee, SANS' lead author for the forensics track invited me to be a panelist for the Summit several months ago. He posted a list of questions that we should be prepared to answer during the incident response panel and gave us the option to come up with our own question based on the Summit's theme.

In a nutshell, the theme of the Summit is that over the last decade forensics and incident response have advanced greatly due to new tools and techniques. What are the new essential tools and methods that incident responders must have or use.

Again, I'm paraphrasing the theme.

From there, I'll be jumping back 100 years, to look at a then emerging high tech field and some highlights (or rather low points) from it's first 50 to 60 years. To see what lessons it might offer us and how those lessons relate to Adam Shostack's and Andrew Stewart's book, The New School of Information Security. Oh, and I've got five minutes to do it so I'm gonna talk fast.

Aside from those five minutes, the Summit is going to be filled with legends in the field(s) and I'm really looking forward to hearing what they have to say.

The Summit is in two weeks and it's going to be amazing. Here's the registration link. Come and join us.

Teaching SANS 508 in San Antonio

2009-05-12T09:13:00.000-05:00

If you're in the San Antonio area and have a need for forensics and incident response training, checkout the SANS Security 508: Computer Forensics Investigation and Response course that I will be teaching during the first week of June.

I have been working incident response and computer forensics for about five years. It is an exciting and challenging field that changes rapidly. I have been reviewing Rob Lee's latest course revision and am amazed at the amount of new material in the course since I taught it last summer.

If you're not in San Antonio, but have a desire to take this training, SANS has launched a "Forensics Tour" that will be bringing 508 to several locations over the coming year. Check the Community Events link over at the forensics.sans.org site for more details.

If you're in San Antonio and want to meetup while I'm there, leave a comment. I'm going to try and attend the San Antonio Hackers gathering the week that I'm there.

Application Security Checklist

2009-04-21T20:13:00.000-05:00

After the 2009 CWE/SANS Top 25 Most Dangerous Programming Errors was released, I started adapting it into a checklist that developers could use during the software development life cycle to facilitate the development of more secure code.

I had reviewed the Top 25 document when it was first released and thought it was pretty good, however, after really diving into it while creating the checklist, I came to appreciate it even more. It's more than a list of the top 25 errors, the document includes guidance for different phases of the software development life cycle, from the design phase through implementation and testing. Some of the entries include code samples clearly demonstrating the errors. It's very comprehensive and any developer would be well served by studying it.

In my seven years as a full time developer, I never worked in an environment that relied on checklists, so I did some searching and happened across an excellent blog post by Brian St. Pierre on the subject.

When creating a checklist, it's important that it be structured such that questions are "yes/no" with affirmative answers indicating a secure situation. Negative answers indicate a problem. This permits reviewers to quickly scan the list for negative answers and follow up on only those issues.

Of course checklists don't guarantee secure software. Developers can lie, or misunderstand, etc. But I've seen firsthand there is value in having developers put their names on a checklist and go through it. Obviously the hope is that they will fully understand the issues and answer truthfully, but just by reading the document, they may learn about the issues and improve their code.

With that, here's the checklist. Please let me know what you think and feel free to use it as you see fit. I owe special thanks to Steve Christey of MITRE for allowing this derivative work.

Fuzzy Wuzzy WebScarab

2009-04-12T09:19:00.000-05:00

I've spent the better part of the last year working on application security issues, including a major effort to implement a secure development life-cycle. I split my days between reviewing source code, application pen testing and developing materials for the SDL initiative.

My primary tools are WebScarab, Burp Suite, RatProxy, a commercial static code analysis tool, a variety of fuzzers, InGuardians' Samurai, Cygwin and some other minor tools.

WebScarab includes a fuzzer plugin that accepts regular expressions or file input for fuzz values and allows you to specify the inputs that you want to fuzz. As an example, let's look at how this works against OWASP's WebGoat, a deliberately vulnerable web application designed for training purposes.

You can use the fuzzer in WebScarab to fuzz POST or GET request parameters. In this example, we'll look at fuzzing a simple POST request. The following screenshot shows the page in WebGoat that we'll be using for this example:

This page has a web form that takes a single search term as input. Nevermind the instructions on the WebGoat page, we're not working the exercise for this example, I'm merely using it to demonstrate WebScarab's fuzzer plugin.

When we enter a search term and submit it to the web server, we can view the request that is sent to the server using WebScarab as shown below:

The top portion of the WebScarab screen shows our request. In the middle pane you can see the search parameter, called "Username" for some reason is being submitted with a value of "lesson".

From the Summary tab in WebScarab, we can right-click on this particular POST request and select "Use as fuzz template" as you can see in the next screenshot:

Once you've selected the "Use as fuzz template" option, switch over to the Fuzzer tab in WebScarab. In the Fuzzer tab, you'll see you can fuzz more than just the POST parameters. Take a look at the next screen shot to see what I'm talking about:

Notice we can fuzz cookies, GET and POST parameters. For this example, we're going to try the Username parameter. The first thing to do is pick our fuzzing Source. We do this by clicking the "Sources" button which brings up the following window:

Notice you can select a file, or enter a regular expression as fuzz sources. Using regular expressions is great for instances when you've got a numeric ID in an input and you want to try a range of other values. Say for instance you submitted a page in the app you're testing and you saw a UID=0013301 in the request. You could create a RegEx entry of [0-9][0-9][0-9][0-9][0-9][0-9][0-9], give it a description (i.e. "0000000 - 9999999"), click Add, then Close and use that regular expression to submit requests to the web server using all possible seven digit UID values. Yes, that's going to take some time.

In this example, however, we're not dealing with numeric input, so this doesn't really apply. Let's choose a file for input. A good file to use for this is All_attack.txt from www.neurofuzz.com. You'll want to run this file through awk to strip out the descriptions of the attacks contained in the file. Here's what the file looks like without being stripped:

A:::Meta-Character Injection
TRUE:::Meta-Character Injection
FALSE:::Meta-Character Injection
0:::Meta-Character Injection(Integer)
00:::Meta-Character Injection(Integer)
1:::Meta-Character Injection(Integer)
-1:::Meta-Character Injection(Integer)
1.0:::Meta-Character Injection(Integer)

It's nice to see what the different types of attacks are, but that's a lot of extraneous data to be carrying around during a pen test. No worries, fire up the following awk command:

awk -F: {'print $1'} All_attack.txt > all_attack

You'll now be left with a file that has one attack per line without the description of the attack. Adding this to your Sources in WebScarab is fairly intuitive so I won't cover it here. Once you've added the source, you can select that source as shown in the following screenshot:

After selecting the fuzz source, click the Start button on at the bottom of the window and WebScarb will start POSTing using your regular expressions or lines from your file as inputs. Yes, you can fuzz more than one field at a time, you can also set the priority so that you can different fuzz inputs for different fields and you can specify multiple sources.

What you won't see in WebScarab, is the response back from the server. For that, specify a save location for your WebScarab session data, open a terminal window in the "conversations" directory where the request and response data is saved and start parsing through the requests and responses using your favorite tools and techniques (grep is your friend) to see what inputs have what effect on the responses from the server. One quick way to isolate these is to look at file sizes for the responses. If you see big variations, you may have something interesting going on.

Sometimes WebScarab gets confused about the number of items it needs to fuzz. For example, the All_attack.txt file contains 362 lines as of this writing. But sometimes the fuzz template will submit a single request and quit. When that happens, reset the fuzz source in the drop down window for that parameter, click in the parameter field and then reselect the fuzz source. I've found that usually fixes the problem.

There are of course other fuzzing utilities available, some stand alone, some not. Burp Suite Pro has a built in fuzzer as well and it may very well be superior to the one in WebScarab. I have more experience with WebScarab so that's what I chose to use for this post. After I get some time in with Burp Suite Pro (gotta buy the Pro version), I'll post an entry on how it works.

SANS Forensics and Incident Response Summit 2009

2009-04-07T22:58:00.000-05:00

I'm psyched to attend (and participate in) the SANS Forensics and Incident Response Summit 2009.

I've been editing and contributing to the SANS Forensics Blog since its inception in 2008, which has kept me in frequent contact with Rob Lee, Principal Consultant for MANDIANT, SANS Faculty Fellow and lead author for SANS' forensics track. As a result, I've known about the Summit for some time and knew there were some good things brewing. Needless to say, I was thrilled to be asked to participate.

I'll post additional information at a later date. It's late, I'm tired and need to be up way too early in the morning.

Computer Forensics Course in Kansas City

2009-04-01T20:24:00.001-05:00

If you live in the Kansas City area and would like to learn or increase your existing knowledge of computer forensics, I'll be teaching SANS' Security 508: Computer Forensics, Investigation and Response at the KU Med Center, two hours a week for 10 weeks beginning May 14 through July 16.

I have taken many information security courses from a variety of vendors and SANS is by far my favorite. Their course are jam packed with useful information that even experienced professionals will be able to immediately apply and 508 maybe one of the most densely packed courses in the SANS curriculum.

Here's a list of some of the items covered:

NTFS, FAT32/16 and Ext2/3 file systems in depth
Acquisition and analysis of memory for responding to live systems
Acquisition of disk images, local and across the network
Timeline acquisition and analysis
A look at the different layers of information on a disk drive
Registry analysis
Application footprinting
A review of legal aspects relating to forensics and investigations
A comprehensive framework of the forensics process
And of course, much more

In addition the course comes with the SANS Investigative Forensics Toolkit, a VMWare image with all the best open source tools you need to conduct forensics investigations and of course the course books.

It's a great course. You're sure to learn loads of useful techniques and meet other info sec professionals from your area. Covering the material over 10 weeks gives you a great chance to digest the material over time. If you're interested, I encourage you to head over to the SANS web site and register. If you have any questions, please don't hesitate to contact me. And if you are interested, but can't take the course, check out the blog (see below) and join the community.

As for myself, I've been conducting incident response and forensic investigations for more than five years. I'm a regular contributor to and editor of the SANS Forensics Blog. I've taught this course before and received high marks from the students. I'll bend over backwards to make myself available outside of class time and invite all students to contact me with questions any time, even after the course is over. I want you to be successful, to learn and to have fun and I won't be satisfied unless those objectives are met.

I must remove this log from my eye

2009-03-13T09:42:00.000-05:00

Yes, apparently, I'm selling snake oil. I wish I could sell more of it or at least find out where all the payments are going.

If you don't know what this is about, well, this is the kind of "beat down" one gets for speaking one's mind on the internet. More details can be found over at Mubix's Room362.com and here.

Mubix is a great guy and a big contributor to the information security community. He is the one behind The Podcaster's Meetups and single-handedly brought us the fire talks at this year's Shmoocon.

I had reservations about contacting Rob to begin with and if I had it to do over again, would choose a different path.

2009-02-19T20:58:00.000-06:00

Steven Branigan has a great blog post about how to pick a pen tester. Branigan's point is that the trustworthiness of the pen testing team should be the number one criteria. I agree, I applaud his post and it prodded me to follow up with this post about something that's been bugging me for the last few years.

I'm amazed at some of the highly regarded people in the information security arena who have publicly bragged about exploiting systems that they clearly did not have permission to exploit. They usually consider these exploits to be harmless pranks on systems of little import, but even the least important systems (kiosks seems to be a popular target) are still maintained by someone in IT somewhere and when those systems are compromised, they still have to pulled, re-imaged and redeployed, this takes time and costs money and there's an opportunity cost and these costs are passed along to the consumers.

As info sec professionals it is our job to defend information systems. During the course of our work, we learn how systems can be compromised. We have a responsibility to use that knowledge for good. If we take that knowledge and use it for pranks or to show off, then we become the very thing we are fighting against. We become a house divided.

It has been said many times before that the difference between a white hat and a black hat is permission. If you don't have permission and you're going around messing with vulnerable systems so that you can brag to your friends about it, then you've crossed the line and it gives a bad name to the entire industry. If you wanna play 733t hax0r and break into things, then you don't belong in the ranks of information security professionals.

All of this is not to say there is not a need for legit vulnerability research, but there is a difference between original research to discover new vulnerabilities, or even non-original research in a controlled environment where you are working with permission to learn a new exploit and the types of "bob stories" I'm referring to above. I'll even grant that there are some "bob stories" that demonstrate a vulnerability without going so far as to exploit the system in question and I have no issue with that. But if your "bob story" involves exploitation without permission, do yourself a favor and keep it to yourself, or better yet, think twice and exercise a little self-control, a little professional ethics and don't cross the line no matter how harmless you think your actions may be.

I'll get off my soap box now.

Shmoocon remix

2009-02-09T00:41:00.000-06:00

Last Friday my alarm woke me up a 3:30 a.m. I rolled out of bed, got dressed and hit the road for destination Shmoocon 2009. This is the first hacker con I've been too since a former employer sent me to Black Hat back in 2006.

Shmoocon exceeded my expectations. Here's a roundup of my experience at the con, no this list of talks attended is not in chronological order, it's been recompiled from my notes, the program and a post-con report I put together for my employer.

I attended a talk by Arshan Dabirsiaghi and Jason Li about AntiSammy. It's an OWASP project that offers an API for validating "rich" user input for sites that allow users to input HTML and CSS, but don't want to be vulnerable to XSS.

They briefly mentioned a tool that they were also releasing called Scrubbr. DBAs can point Scrubbr at MS SQL or MySQL databases and it will find fields that contain persistent XSS.

Julia Wolfe of Fire Eye spoke about the company's work to bring down the Srizbi botnet. It was obvious from her slides that they had done some really great analysis of the malware and inner workings of the botnet, very cool stuff.

Shane Lawson gave a hilarious presentation about defeating Kwikset's SmartKey. This one made me want a lock pick set to call my own.

I also introduced myself to Paul Asadoorian and Larry Pesce of PaulDotCom fame. Larry and David Lauer of the Security Justice Podcast showed off their ShmooBall Launchers.

After the days talks were over, I headed over to hear the Podcasters Meetup. It was a great time and lots of prizes were given out. I introduced myself to Mubix, he did a great job arranging the meetup and the after hours talks. I hope this can become a regular part of Shmoocon.

Following the fire talks, HacDC hosted a great after hours party that was around two miles from the hotel. I'm a regular walker. I try to get out for at least a three mile walk every day so I knew walking to the party would be a snap, navigating the whacked out streets of DC was a bit of an adventure. I live in a part of the country where the streets are on a nice east/west, north/south grid.

The party was in an old church and featured an open bar in exchange for donations at the door. Of course the highlight for me was meeting CrucialCarl, gattaca, rybolov and myrcurial at the bash. Rybolov brought his didgeridoo and flabongo to the party.

Walking home from the party was far more adventurous than walking to the party. I tried to walk back without the aid of my iPhone. Of course I missed a turn and ended up going a few blocks too far to the south, but I quickly got back on course and found the hotel without incident. To walk a city is to know a city.

On Saturday I sat through another great list of talks. Enno Rey and Daniel Mende of ERNW gave a pair of great talks. The first was on novel ways to build botnets. One technique involved finding and taking advantage of all the ignorant fools who have snmp devices unprotected on the internet with private strings exposed.

Their second talk of the day was on attacking internet backbone protocols. It's amazing how the same kinds of security issues that plague software development also affect core internet protocols and they are all based on misplaced assumptions about trust.

Shawn Moyer and Nathan Hamiel gave a hilarious talk about problems they've found in social networking sites. These two obviously love what they do, are good at their work and could likely make a second career in stand-up.

There was a lot of hallway talk and online bitching about Jay Beale's and his vaproware, Middler (yes it's out now so stop complaining). His talk was good and it was awesome to see this tool has finally been released, even if it is a bit rough around the edges. I have downloaded the code and have been working through it.

Prior to Jay's talk, I finally met hevnsnt and surbo from i-hacked. We had a nice lunch at the Indian restaurant a block from the hotel. Great couple of guys and I hope I can get more involved in all the stuff they are doing in the Kansas City area. If you haven't purchased a raffle ticket to help them cover the rent for their hacker space and possibly win yourself a MacBook, well, what are you waiting for. Go do it now!

I attended a few other talks, but am too exhausted now to keep going. I met a bunch of interesting and wicked smart folks that I've only known virtually and saw some old friends including Ed Skoudis and Kevin Johnson of InGuardians. I introduced myself to Johnny Long early on Sunday morning when he was setting up. He's everything everyone has ever told me he is, approachable, friendly and seems genuinely interested in what others have to say. I hope to get more involved with Hackers for Charity.

I paid my own way to Shmoocon and used vacation to cover my time off. I took good notes and when I returned to work, I wrote up a little two page report for my boss of all the things I learned or thought might help out our office. As a result, she rejected my time sheet and told me to replace my vacation days with non-project training. It would have been nice to get reimbursed for my travel and lodging, but I wanted to go to Shmoocon for myself and I'm glad I did and I'm looking forward to next year.

Incident Response

2008-11-07T10:10:00.000-06:00

Canadian expat, published info sec author and Bermudan beachcomber, Andrew Hay recently posted a question over at Michael Santarcangelo's Security Catalyst Community Forums asking about a framework for incident response for a bank of several hundred employees.

I gave my answer largely based on Ed Skoudis' outstanding SANS Security 504: Hacker Techniques & Incident Handling course.

I won't rehash that answer here, but I will provide some additional insights. I mentioned in my post that I used to work in info sec on a large, unprotected higher ed network. Enterprise uniformity was non-existent. The academic freedom that makes universities vibrant and interesting makes life hell for info sec personnel. If managing developers is herding cats, running an info sec program in higher ed is like being the poop-scooping clown at the back of the cat herder's parade. Hm, need a better metaphor.

Given the nature of our environment and the constraints placed on info sec, incident response was a regular activity. During my time working in academia, I responded to hundreds of incidents, thank you Blaster, Sasser, Zotob and the countless number of postcard.gif.exes.

If you're creating incident response policies and procedures, you want to be careful that they are not overly specific. Since incidents vary widely, you don't want to be constrained by a short-sighted plan that didn't account for this specific incident. If your plan states you won't pull the plug on a system without approval of the system's owner and an incident is occurring where plug pulling is needed, but the owner is unreachable, you're damned if you do and damned if you don't. Your policies and procedures need the right amount of flexibility.

If you're an info sec manager, you'll want to run interference for your incident response team. Send your IR folks out in pairs or larger teams. While one person works the incident at the keyboard, another can talk to the system's owner or the manager of the affected department. The handler at the keyboard is going to need to concentrate and that can be difficult with people walking all over you.

On a technical note, if you're going to image the system(s) in question, by all means make an image of the RAM for later analysis. There's more and more evidence of malware that remains memory resident only and if you don't grab the contents of RAM, you may not find all the evidence you need from a hard drive image only.

There's much more to be said about IR. It's a big and constantly changing field requiring practitioners to stay current.

Computer Forensics, Investigation and Response

2008-10-25T21:50:00.000-05:00

I'm excited. For 10 weeks this summer I was privileged to teach SANS Security 508: Computer Forensics, Investigation and Response via the Mentor program. It is one of my favorite SANS courses for its depth and the extensive hands-on exercises. Unlike other forensics courses that teach specific tools without getting into what's going on behind the scenes, this course pulls back the curtain with an in depth look at different file systems and how they store and organize data on disk.

Once we've covered the foundational materials we introduce a comprehensive methodology that covers all the important aspects of conducting a successful investigation. There's even a day focused on legal issues.

If there's one problem with the course, it's the sheer volume of information to be digested. One nice thing about covering it over 10 weeks, as opposed to in six days, is that you get more time to take it all in, try things out, absorb the content and experiment with the tools and concepts.

I'm excited because I get to do it again starting in January. Full details are available at http://www.sans.org/mentor/details.php?nid=14464. If you live in the Kansas City area and are interested, please check it out. If you know someone else who may benefit from this, please spread the word.

As seen on Twitter

2008-10-11T08:28:00.000-05:00

# mdowd @alexsotirov The first year online voting is allowed, a cartoon character is going to win by 43 billion votes about 4 hours ago

# mdowd @alexsotirov Because I'm not interested in the common good! about 4 hours ago

# alexsotirov Why is it that all these source code audits of voting machines are done by university professors instead of Mark Dowd? about 4 hours ago

SANS' Web App Pen Testing In Depth Day Four

2008-10-07T22:17:00.000-05:00

Day four rocked. It was exploit day and sure all the stuff leading up to exploitation is important, but there's nothing like the joy derived from breaking stuff. And the exercises were more complicated, which I think it good and I may be wrong about this, but I think there were more exercises on day four.

I played around with some interesting tools that I hadn't used before, BeEF for one, and Kevin talked about some tools and ideas that are being developed by him and his colleagues at InGuardians. What a great bunch of minds at InGuardians. I aspire to be like the folks in that company and to work with a similar group of people.

The class wrapped up with an overview of the materials and the process. I'm excited for Kevin going forward. He's put together a good course and it's only going to get better when the six day version comes out.

I stand by my earlier statements about Kevin. He's a great teacher and a judging from the two nights I had the good fortune to have dinner with him and hangout for a bit, he's a quality human being. I've had some brilliant instructors over the years and they knew it and the result was that they were not very approachable. Kevin is a fantastic instructor and two days in a row invited anyone from the class to join him for dinner.

If you're looking to get started in web app pen testing, or you've been doing it for a little while and aren't sure about your methods, I strongly recommend this course. I had some experience with web app pen testing prior to taking the course. The result for me was that the first couple of days were mostly review with a few new nuggets here and there, but day three and especially day four really broke some new ground for me. The entire course also validated my own methods and as an aspiring instructor, it was great to watch Kevin teach the class. He's a natural and I am looking forward to seeing him again at a future con. I hope I can make it to Shmoo in February and maybe catch him there if he attends.

Day Three of SANS' 542: Web App Pen Testing In Depth

2008-10-06T01:37:00.000-05:00

Day three was a run through of discovery. Discovery is the process of finding vulnerabilities in the web application, but stopping short, for the most part, of actually exploiting vulnerabilities.

By the way, it's really late and I should be asleep, so I'm keeping this post short. Or I intend to.

We looked at information leakage, this is one of the things I find most often in my own testing. Developers allow their applications to throw errors back to a user and the errors leak information about the implementation of the application, such as what OS, backend DB or other components of the system. Yes, this is bad. You should be returning generic error messages. Preferably something that has the right look and feel for your application. Many developers I work with on a regular basis simply redirect the user back to the start of the application when the app throws an exception. This is not as bad as leaking information, but it sucks for usability. Don't do it that way.

We looked at username harvesting. This is something I find quite a bit in my work. And it's a difficult problem to overcome if you spend much time thinking about ways to mitigate it. If you have an app where people can register for a new account, it's hard to prevent username harvesting for obvious reasons. Password resets, security questions and the like are another area where username harvesting is pretty common, but generally is more preventable. Account registration and creation is the biggie.

I brought this up with Kevin and he had an excellent suggestion. Don't prevent it. Detect it and block the attack. I'll be writing this up as a recommendation in the future.

We looked at fuzzing applications using the Burp Suite, talked about Absinthe. I wish there would have been an exercise for Absinthe. I have it installed on my pen testing box, but haven't used it yet.

Greasemonkey was introduced. I love Greasemonkey, though I've never used it for pen testing. I find it really useful for adding functionality to web interfaces. It rocks.

The last part of the day was a review of some of the newer developments on the web; namely Web Services Definition Language, Universal Description, Definition, Integration Specification (UDDI), Simple Object Access Protocol, AJAX and JSON. Frankly, I wish we could have spent an entire day on these areas alone.

You've all heard of Web 2.0. AJAX and JSON are two of the core components that drive Web 2.0, but many larger enterprises are only now beginning to role them out so many web app pen testers don't have much experience with them, including yours truly. I could use more info on these technologies and have it on my list to find out as much as I can in the coming weeks.

All in all, another good day. Again, I wish the course had more hands-on exercises and as I've mentioned previously, I know it's coming in the six day version of the course. In fact, I had dinner with Kevin Johnson, the author of the course and a couple other students and Kevin talked about how day six of the course is going to be a full on web app pen test exercise from start to finish. If it includes all the whiz-bang Web 2.0 aspects, that will be really beneficial.

Now if you'll excuse me, I should have been asleep a couple hours ago. Good night now!

Day Two: SANS Network Security 2008

2008-10-04T23:07:00.000-05:00

Today was day two of SANS' Web App Penetration Testing In-Depth. The focus of day two was information gathering.

Scouting your client is an essential part of the process and the course presents methods and tools for accomplishing this. Google is your friend. Johnny Long's Google Hacking For Penetration Testers will be a trusted companion. Some less well-known, but highly useful methods were covered. If you've studied pen testing, you may have knowledge of these. If it seems I'm being vague, that's because I am. I respect the work that Kevin has put into developing the course and I'm not going to give it all away.

I do wish the course had more hands-on with some of the tools that are presented. I know from talking to Kevin that the course is going to be expanded. Perhaps the expanded version will include more exercises with some of these tools.

After gathering useful info about the company, we drill down and gather details about the technologies that are being used. The usual tools for gathering information about systems are covered, but there were a few that I have not used including RSnake's Fierce.

There was a discussion of some issues relating to web application server architecture and some of the caveats that can throw off a pen tester and ways to work around those obstacles.

We continued to drill down, from the servers to the apps on those servers and ways to gather useful information about those applications. Most of the tools discussed were ones I'd had experience with before. One exception was OWASP's DirBuster. I'd seen it before, but never bothered to try it out. Now I have and will incorporate it into my testing.

All in all, another good day in class and I'm really looking forward to tomorrow when we enter the discovery phase where we will uncover weaknesses in the app.

SANS Network Security 2008

2008-10-03T21:42:00.000-05:00

I don't want to come off like a fan boy, but I've been taking training in the information technology arena for more than a dozen years and from a variety of different sources. SANS is better than any other organization I've trained with. In the interest of full-disclosure, I have participated in SANS' Mentor program, but am not an employee nor do I have any affiliation beyond that.

I am currently in Las Vegas at Network Security 2008 attending Web Application Penetration Testing In-Depth developed and taught by Kevin Johnson of InGuardians, developer of BASE, Samurai and many other Open Source projects.

I wasn't sure I should take the course. I've been doing web app pen tests for a while. By no means am I an expert and I don't claim to know all there is to know, but I wasn't sure I would get enough from the course to make it worth my while. I'd say on a scale of one to five, five being an expert, I'm probably almost a four. You should know that one of my many flaws is that I consistently underestimate my abilities.

Day one didn't teach me very many new things about web application pen testing, but there were a few nuggets. However, based on day one, I am confident that over the next three days I will pick up many great insights that make me more effective.

Johnson has put a tremendous effort into the course materials and he may be the best instructor I've ever had. He has a very friendly and knowledgeable approach. He's clearly a subject matter expert, but he has the right amount of self-effacing humor.

Based on what I've seen thus far, being in this course is going to have two great benefits. I will learn to be a better web app pen tester and will learn how to improve my teaching skills.

Feldman's product or idea maturity model

2008-10-01T12:47:00.000-05:00

I was listening to Gary McGraw's Silver Bullet Security Podcast Show 002 where Dan Geer was the victim.

If you haven't listened to the Silver Bullet Podcast, it is a series of interviews with information security luminaries. I find most of the guests to be fascinating and Geer was obviously no exception. He had many great things to say.

One of the things Geer mentioned during the show was that he used to attend talks by Stu Feldman a computer scientist forged in the bowels of Bell Labs, what an amazing place that must have been to work. Feldman is the creator of the make utility and is currently the VP of Engineering at Google.

Apparently Feldman had this concept for evaluating the maturity or quality of an idea, concept or product. After the show, I searched for it online and couldn't find it, so I thought I'd share it here in hopes that maybe Google will index it and others could find it later and know who to attribute it to.

Feldman's model has five levels so they can be tracked on one hand. Here it is:

1. You have a good idea.
2. You can make it work.
3. You convince a gullible friend to try it.
4. People stop asking why you're doing it.
5. People start asking others why they aren't doing it.

That's it Feldman's method for evaluating the maturity of a product or idea. With apologies to Mr. Feldman for publishing this without his permission.

Strategic thinking and doing

2008-09-09T22:02:00.000-05:00

At my day job, I'm responsible for putting security into the Software Development Life Cycle. Unfortunately, there's not much of an SDLC to work with.

While the organization has an architect, he's too busy with short-term projects to focus on longer term strategy. His attention is on an encompassing portal project, while infrastructure issues such as developing standards relating to languages, tools and processes go unattended.

No two development teams work the same way. There's no central repository for code that can be shared across the enterprise. Even within the same team, developers don't have access to all of the same tools.

Few of the external facing web pages adhere to accepted web standards and even the branding is inconsistent. These are not security issues, but they reflect an overall pattern that permeates the organization's development efforts.

Part of me wonders if adapting the Fixing Broken Windows crime fighting approach might help us clean up all aspects of our development process the same way it helped clean up the streets of New York.

Freedom and security

2008-09-02T22:19:00.000-05:00

If you want to see something about the nature of the relationship between freedom and security, go search flickr.com for images tagged with "freedom" and "security."

SANS Network Security 2008

2008-08-29T22:33:00.000-05:00

I will be heading to Las Vegas for SANS Network Security 2008. I will be attending Kevin Johnson's Web Application Penetration Testing In-Depth course. Johnson is of Intelguardians fame, the founder of the Basic Analysis and Security Engine project. BASE is a Snort analysis database and front end. Johnson is also the driver of Samurai, the web application penetration testing LiveCD I wish I'd had three months ago so I would not have cluttered my base OS with so many tools.

I'm psyched about going to NS2008. The last time I was in Vegas was for Black Hat 2006.

If I've met you online via Twitter or through the PaulDotCom.com IRC channel and you're going to be in Vegas, let's meet up and grab a beer. And if you're Kevin Johnson or Ed Skoudis, I've already promised you a round.

Security at your table?

2008-08-22T20:20:00.000-05:00

Information security professionals are a disgusting lot. They chew loudly and with their mouths open. They slurp when they drink and talk with their mouths full. They make no attempts to politely cover belches and they fart at the table. They don't wash their hands before or after eating, in fact, they rarely bathe at all.

How else do we explain the fact that security is often not invited to the table? Or if they are allowed a spot at the table, it is all too often so that they can bus the dirty dishes and clean up everyone else's mess.

If your organization is serious about security, give infosec a seat at the table at the start of the project planning process. Much has been written about the cost of adding security or fixing bugs late in the software development process. This is not new information.

If your company wants to develop more secure software, bring qualified security personnel to the table during the requirements gathering phase. Ask them to contribute to the project from start to finish. They should have input every step of the way. In addition to reviewing the customer's functional requirements, they should provide input on the system's security requirements.

After requirements have been gathered, include security in the planning phase of your project. Don't just ask security to review your plans, ask them to contribute to the planning process. Invite them to the planning meetings.

Ask security to help you with testing along the way. If you have a static code analysis tool use it early and often. You may save yourself days or weeks of refactoring if you discover insecure coding techniques early in the development process. Whether you have a static code analysis tool or not, you need to include security in your code review process.

Finishing your code and handing it off to security for a comprehensive review at the end without their involvement along the way is better than nothing, but it is less than ideal. Often as a project nears completion, delivery schedules are being made. Too often these delivery schedules are made without input from the security team. Developers and their managers underestimate the amount of time that will be required to review code. Do not make this mistake in your organization. Do your code reviews along the way and include security in that process as you go.

Once your development nears completion, begin planning your application penetration test. Obviously security needs to be included in the planning for the penetration test. I recently worked an application pen test where I was tasked with testing some changes to an existing application. Testing the changed functionality required three different types of accounts, yet when I was brought in to look at it, I hadn't been given a single account in the system. Due to the nature of the application, getting accounts created and properly setup took several days. All the while the clock was ticking on a scheduled delivery date. Fortunately for this organization, the test was completed successfully a day before the scheduled delivery date.

One more thing you should know, a successful penetration test will prove that your application has security problems. A failed penetration test does not prove that the application is secure. The best chance of building secure systems is to invite security to the table early and keep them engaged throughout the process.

Don't worry, when your system is compromised and someone makes a mess of things, you can still call security to have them clean up the mess.

Touch on Windows via PowerShell

2008-08-15T07:11:00.000-05:00

A forensic investigator recently inquired about a touch equivalent for the Windows environment. If you don't know touch is a command in the *nix environment that allows you to modify file timestamps arbitrarily.

My first thought was that maybe wmic could accomplish the task. Turns out wmic can only read timestamps, not set them.

More digging revealed that Microsoft's PowerShell could be used to modify file timestamps.

Below is the nitty and the gritty.

From within powershell:

$(Get-Item ).creationtime=$(Get-Date "mm/dd/yyyy hh:mm am/pm")
$(Get-Item ).lastaccesstime=$(Get-Date "mm/dd/yyyy hh:mm am/pm")
$(Get-Item ).lastwritetime=$(Get-Date "mm/dd/yyyy hh:mm am/pm")

There are also utc timestamp attributes (CreationTimeUtc, etc.). I
haven't touched (no pun intended) those.

Here's a sample run from my PowerShell prompt (PS>):

PS > date

Thursday, August 14, 2008 9:38:47 am

PS> echo > test.txt

PS> dir
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 8/14/2008 9:38 AM 0 test.txt

PS>$(get-item test.txt).lastwritetime=$(get-date "08/31/2012")

PS>dir
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 8/31/2012 12:00 AM 0 test.txt

You can use these commands to change timestamps such that their
CreationTime is later than their other timestamps.

Fun stuff.

Blended attacks for pen testers

2008-08-13T11:47:00.000-05:00

When Dan Kaminsky microblogs about a security advisory, CVE-2008-3280 and says "brainmelt", you have to think it's something particularly good. He did destroy the internet after all.

If you haven't read CVE-2008-3280 it discusses findings by Ben Laurie and his team at Google in cooperation with Dr. Richard Clayton. In short, the advisory discusses awesomely powerful blended attacks that leverage Kaminsky's DNS findings and the entropy issues Debian suffered earlier this year and the lack of CRL checking by browsers. See (CVE-2008-0166).

Will this be the year of the blended attack? Recall CVE-2008-2540, the blended attack that relied on Safari's saving downloaded files to the desktop and the way Windows desktop deals with executables.

Along these lines, I am looking forward to the insights Ed Skoudis and Kevin Johnson will share at SANS Network Security 2008. Skoudis and Johnson are teaming up to deliver the keynote titled "The Ultimate Pen Test: Combining Network and Web App Techniques for World Domination."

In my own experience conducting web app pen tests, I've found command injection flaws that allowed me to execute arbitrary system commands as the Apache user. Granted running commands with Apache's privilege level isn't as good as being root (unless the box is misconfigured), but the Apache user can cat /etc/passwd, see who frequent users are via the last command, or depending on egress filtering, may be able to run traceroute from the web server to help map the network from the inside out, or download a pen tester's agent to facilitate deeper penetration. Ahem.

Or consider a web application that contains a Cross Site Request Forgery (CSRF or XSRF depending on who you ask) vulnerability. If such a flaw exists in the web based management interface for a network security device, we have a pen testing situation that will benefit from the skills of both the web app pen tester and the traditional pen tester. Sharpen your spears for a little targeted phishing. Use Google to find postings by the firewall administrator for the organization. What are the odds that admin will be logged into the firewall web gui for hours at a time each day? Craft a good email message with a tempting link for that admin, get him to click it while logged into the vulnerabe web app and you're way.

If someone like me with limited pen testing experience can think up simple ways like this to use a web app pen test as a force multiplier for a network pen test, imagine what Skoudis and Johnson, both experts in the field will have to say on the subject. Their keynote in Vegas will be one of the best infosec talks of the year.