Raesene's Ramblings

Hello and welcome to our new blog.  We've merged in the content from Rory's old blog and spruced it up with the excellent Centita theme.

Sometimes when you're testing it's good to be able to quickly get a feel for where to focus your attention or to get an overview of all the ports you've got open, so you can be sure you investigate all of them. Once you've done several scans as part of a job, you end up with a stack of nmap and nessus output files, and it can be hard to keep an eye on exactly what's been found so far, and it's good to have a way to just get the facts.
As a result a lot of testers will have scripts to help parse collate output from common tools like nmap.
They tend not to be the prettiest code in the universe or to produce lovely management friendly reports, but handy nonetheless.
Having set expectations about the quality of this code :) Here's a couple of scripts which may come in useful for testers in managing some of those xml report files.
NMAP Auto Analyzer can parse a single nmap xml file or a directory of nmap xml files and provide a concise report on ports open across them.
Nessus Auto Analyzer somewhat unsurprisingly, does the same job for .nessus report files (v2 only at the moment)
Both have reasonable help files, so should be fairly straightforward to use, any questions/queries welcome either in comments or in e-mail (rorym at mccune dot org dot uk)

#{note_data['technical_notes']} eos end

This just puts together the body of the note description for each finding, as one long string.
There's obviously a lot more that could be done with this (like better error handling and writing tests) but with those files complete, the module should work ok and you should be able to import vulns from your database directly into Dradis using the "import note" feature.
I've put a copy of the code for the plugin up here, in case it's helpful :)

One of the main tools that I've found useful in pen. testing is the Dradis Framework, it's a good way of keeping track of findings and notes during a test and I've also found it's template feature is good for keeping a list of things to remember during a test.
One of the features available in Dradis is import plugins. This lets you create a link to an external information source, such as a OSVDB or a database of vulnerabilities.
Having a database of vulnerabilities or findings can be pretty useful in cutting down the time required for reporting on a test as you can keep standard wordings in place (who really wants to write the same section about preventing XSS more than once!).
So recently I knocked up a simple vulnerability database to link in to Dradis and I thought it might be of use, so here's the process.
Creating the App
We're going to use Ruby on Rails for this as it's nice and easy to develop for (as you'll see) and also that's what Dradis is based on, so makes sense to keep all the coding in the same underlying language. Also rails apps are very portable, they're basically contained within a single directory structure, so it's relatively easy to move them from place to place.
Before starting the application, there's the usual pre-req's. I'm using Ruby 1.9.2 and Rails 3 so having those installed is a good thing. If you're using Linux then it's helpful to have RVM working as some distros don't have ruby 1.9.2 packaged up as yet.
once you've got the pre-req's working, we can start by creating a rails app

rails new vulnlist

This creates a new application called vulnlist and adds all the standard rails files in.
Creating the Scaffold
Once we've got the app created, we can use rails scaffolding to quickly create the basic structure for our app. The web pages that scaffolding creates aren't the most pretty, but they'll do for now.
With the scaffold we can specify what fields we want to create in the database and also what data types those fields are.

rails generate scaffold Vulnerability title:string test_type:string description:text remediation:text technical_notes:text severity:string owasp_reference:string

Once we've completed this we can look at the basic app by setting up the database with

rake db:migrate

Ensuring that all our gems are installed ok with

bundle install

and launching the app

rails server

At this point browsing to http://127.0.0.1:3000/vulnerabilities should show a blank page with our fields in it. From this page we can create new vulnerabilities and edit or delete existing ones.
Now that we've got this basic structure setup it's worth using git to keep a handle on the source code. On Linux the procedure for this is pretty easy
If you've not already got it installed

sudo apt-get install git-core

then in the root of the application

git init
git add .
git commit -m "Initial Commit with Scaffold"

Having git running on the app will make it pretty easy to revert any mistakes made along the way, as long as we've done regular commits.
Setting up the Searches
So far we've got a basic structure in place and can do the basic Create, Read, Update, Delete cycle on our data. However for the dradis integration, what'd be useful is if we could search for vulnerabilities using various criteria and have the results returned to Dradis.
This turns out to be relatively straightforward. First what we need is a new action in our controller. Opening vulnlist/app/controllers/vulnerabilities_controller.rb we can see the existing actions that we've got for the application.
What we need to do now is add a new action to allow for vulnerability searches.

def vuln_search
case params[:search_type]
when "description"
@vulnerabilities = Vulnerability.where("description like ?", "%"+params[:query]+"%")
when "owasp"
@vulnerabilities = Vulnerability.where("owasp_reference like ?",params[:query]+"%")
when "severity"
@vulnerabilities = Vulnerability.find_all_by_severity(params[:query])
when "test_type"
@vulnerabilities = Vulnerability.find_all_by_test_type(params[:query])
end
respond_to do |format|
format.xml {render :xml => @vulnerabilities}
format.json {render :json => @vulnerabilities}
end
end

This defines a new method called "vuln_search" which takes two parameters, search_type and query. The search type parameter lets us pick from different finders. Rails provides access into the application database via ActiveRecord and this just uses a couple of the finder types for different parameters. Where the query is going to be one of a number of fixed values like "severity" which will be something like high, medium or low, we can just use a standard find_all_by_ approach, but where it's a more free text style search, we use Vulnerability.where and pass in the query parameter that way.
The respond_to block is a really nice feature of rails. By adding in the two lines for :xml and :json rails wires up responses so that we can get the data out in those format, no additional code required.
Now that we've got the basic code in place, we just need to modify the rails routes so that the application knows how to access our new method.
This is done by modifying the vulnlist/config/routes.rb file, and adding the following code

controller :vulnerabilities do
get 'vuln_search' => :vuln_search
end

At this point, we've got the application basically working. If you put in a couple of test findings, then you should be able to go to http://127.0.0.1:3000/vuln_search.xml?search_type=Severity&Query=High for example and get some XML data back.
Tidying up
Now that we've got the basics working, there's a couple of additional steps that it's worth looking at to tidy some things up.
Selectors
First off, we'd like some of our fields (OWASP Reference, Severity and test type) to be one of a number of defined values. The "proper" way to do this would be to create additional models for these and link them in to the main vulnerabilities controller, but there's a quicker way which is probably going to work well enough for our purposes.
Opening up vulnlist/app/models/vulnerability.rb we can specify some Constant values for these settings

TEST_TYPES = ["Web Application","Windows Server","Unix Server","Wireless","Web Server","Oracle","MS SQL","MySQL","DB2"]
SEVERITY_LEVELS = ["Critical","High","Medium","Low","No Impact"]
OWASP_TOP_10 = ["A1 - Injection","A2 - Cross Site Scripting (XSS)","A3 - Broken Authentication and Session Management","A4 - Insecure Direct Object Reference","A5 - Cross-Site Request Forgery (CSRF)","A6 - Security Misconfiguration","A7 - Insecure Cryptographic Storage","A8 - Failure to Restrict URL Access","A9 - Insufficient Transport Layer Protection","A10 - Unvalidated Redirects and Forwards"]

Then we can modify the form that the scaffolding process created to use these arrays as a select list. The form is found in vulnlist/app/views/vulnerabilities/_form.html.erb. In that file we just need to replace the "text_field" lines for those three fields with the following select lines

 "Select the test type" %>
 "Select the severity level" %>
 "Select the appropriate OWASP top 10 reference" %>

This picks up the Constants from our model and helps keep the data consistent.
Localhost Only
As you'll have noticed with this application, there's pretty much no security whatsoever. At the moment it's setup as a personal database only and isn't suitable to be on any kind of network. Adding that security isn't too difficult with rails, however it's not really a problem for the basic use case that we have here. Both the vulnerability list and the dradis installation only need to listen on the localhost.
Configuring rails to only listen on the localhost (as opposed to specifying it on the command line) is a bit hacky, but here's a way to do it based on this post and this dradis change . We need to modify the vulnlist/script/rails file and add the following lines

require 'rubygems'
require 'rails/commands/server'
require 'rack'
require 'webrick'
module Rails
class Server  3003,
:Host => "127.0.0.1",
:environment => (ENV['RAILS_ENV'] || "development").dup,
:daemonize => false,
:debugger => false,
:pid => File.expand_path("tmp/pids/server.pid"),
:config => File.expand_path("config.ru")
})
end
end
end

This also moves the application off the default port of 3000, to a new one of 3003 which hopefully shouldn't clash with other services.
Default Routes
At the moment if we visit the root page of our application, now at http://127.0.0.1:3003 we get the default rails welcome page. What'd be nicer is if we were re-directed to the vulnerability listing automatically.
That's easily done with two steps. First edit the vulnlist/config/routes.rb file and add the line

root :to => "vulnerabilities#index"

then delete the file vulnlist/public/index.html file.
Summary
So at the end of this first part we've created a basic vulnerability database which we can search easily on a number of parameters.
The next step is to create the dradis plugin to hook the two together, which as I'll cover next time is a reasonably easy thing to do.

I've just started a new role as a director at 7 Elements. We're providing technical security consultancy and penetration testing services, focusing on the scottish market.
As part of that we've started up a blog here to talk through some of the ideas we've got for approaching security and testing in a pragmatic way.
I'm planning to keep this blog running for now (not that you could tell from the level of posts!), but more of my security/testing stuff will probably pop up on the 7 Elements blog...

I had some cause to do some wireless work recently, which got me interested in doing some more war-walking (and hey, the weathers actually been nice enough to make it pleasant recently).
It was interesting to see the density of wireless networks in the suburban area near where I live, a quick 30 minute walk can easily pick up several hundred APs. Also some of the stats on encryption were interesting with about 25% of networks either using WEP or having no encryption at all, so still rich pickings there for anyone who wants free access or to attack some home networks directly.
I also did a bit of scanning with my N900 in Glasgow, near the apple store and noticed they've got an awful lot of clients connected to their unencrypted wireless networks there (~ 260 client spread over 3 APs), hope everyone is using VPNs or SSL only sites ;op
Also couldn't find something to do the analysis the way I was looking for it, so I knocked up a quick script in ruby to analyse the .netxml output files from kismet.
It's available here . It needs ruby, rubygems and nokogiri to work. worth noting that on linux installs you'll need some xml parsing libraries installed before installing nokogiri (libxslt libxml2 libxml2-dev)
Basic syntax is very straightforward.
./kis_analysis.rb -f [netxml file] -r [report name]
you can add -g if you've got GPS data to add links from each network to a google maps point and -m to draw a map of all the networks seen.
Any feedback/comments welcome either on the blog or to rorym@nmrconsult.net

One of the aspects of the move to cloud computing I find most interesting is the new and emergent risks that come with the move of services from a traditional networked IT environment, to being hosted "out in the open" of the cloud.
Whilst attention gets paid to some of the technical risks, I don't think there's been a lot of focus on some of the more procedural/human aspects of it.
One example is the visibility/effect of configuration mistakes. In a traditional IT environment, mistakes can be partially contained by the network perimeter (albeit that containment is usually weaker than it used to be).
If someone makes an access control change which allows anonymous access to data, that mistake is likely only to be exploitable and visible to a limited group of people.
With the move to Cloud computing though, that same mistake could be instantly visible to the whole world and all it's attacker communities.
A really good example of this comes up in a vulnerability found by Jonathan Siegel (background story here and here).
In essence the problem seems to be that users of Amazon Web Services have made access control errors which set disk snapshots to be publicly available to everyone in a given region, and in the examples Jonathan gives this has included a database of user accounts for a web service and a full copy of a news services' web site.
So what would have likely been a relatively minor access control issue in an Internal network setup, becomes a situation where all the data in question should be considered compromised.

Most of the questions I got after my talk were around how people can look to secure their application. I mentioned a couple of sites and it's probably worth expanding on the points made.
Web Application Security
For people looking to understand how to secure their web applications, in my opinion the best source of free information is the OWASP project . It's an umbrella for a number of web application security efforts.
They've got a wide variety of projects which target management, security testers and developers, but some of the best places to start for developers are the Owasp developers Guide which covers a wide range of topics about secure development in a fairly generic way, and there's also the Ruby on Rails Security Guide here or here which covers rails specific topics.
Both of those documents are relatively large but well worth delving into. For a quick few of the "most serious" web application security issues, OWASP also has the Top Ten project. The latest version, current in RC1 has some good summaries of the top risks, and is a good place to start getting an idea of the areas to be addressed...
There's also a newish book dedicated to the topic of security on rails. I picked up a copy at the conference (I'm a sucker for the O'Reilly stand!), so I've not had time to get all the way through it, but first impressions are that it'll be very good, with concrete examples of broken rails applications and how to fix them.

Just had the first day of the Scottish Ruby Conference. The venue was awesome, there'll doubtless be lots of good pictures up on places like flickr in due course, but here's a couple I snapped with my Nokia n900. The three track rooms were the Conference Hall, the Great Hall and my personal favourite the "new" library.
The talks were interesting as always, personal highlight for me had to be Jim Weirich managing to go from particle physics to functional programming seamlessly in a single talk!
my talk was on breaking things with ruby and was a pretty quick look at the world of penetration testing and some of the cool projects (primarily Metasploit) which use ruby in this area. My slides can be found here and if people are interested in more information on the topics I covered here's some good linkage to start with.
Metasploit
The Main Metasploit Website . This is the best place to start. There's also a wiki and source code repository on their redmine site here
Metasploit Unleashed. This is a free training course from Offensive Security which offers a good place to start with Metasploit, and covers a lot more ground than I could today.
Blog-wise there's a couple of places I consistently find good information on Metasploit. Carnal0wnage's blog and the Dark Operator blogs have lots of posts on what can be done with Metasploit, as well as the Official Metasploit Blog of course.
Also on twitter following @hdmoore, @carnal0wnage and @egyp7 is a good place to start.
Lastly if anyone's looking for videos demonstrating Metasploit, there's quite a few on Security Tube amongst other places.

If you've not already heard about it, the Scottish Ruby Conference is coming up in March in Edinburgh. There's a really interesting set of presentations lined up for this years conference, and the hardest thing, I reckon, will be picking between the three tracks!
I've been very lucky to get my talk accepted for this years conference, so I'll be talking about "Breaking things with ruby". It sits very nicely for me at the corner of two of my main computing interests. Using ruby and ... well breaking stuff!