The ‘file’ command is a nice tool. It has a database of filetypes and “magic” numbers which correspond to offsets and values within a file and are used to hazard a guess as to what type of file it is. On my system, the /usr/share/file/magic database has 13474 lines in it. Quite a bit of knowledge about filetypes at your fingertips!

To use it simply:

$ file <targetfile>

Example:

$ file /pictures/nice.jpg

/pictures/nice.jpg: JPEG image data, JFIF standard 1.02

or

$ file ./unknown

./unknown: VMS Alpha executable

What happens when dealing with “unknown” file types that may not be accurately described by the “file” command’s knowledge of filetypes? Or, what happens when a file contains many other files within it that we can easily get to? We can attempt to peer inside an unknown container file and find what types of other files it is made of… by sliding along the file and comparing every offset to the magic database.

Luckily, there is a python binding the “magic” database.

# apt-get install python-magic

And a handy example is included in /usr/share/doc/python-magic/examples/example.py.

Excellent. This is just what we need. Our algorithm is simple. Loop over each offset in the file and see what python-magic thinks it is. Interesting offsets can then be identified and extracted for further analysis.

Here’s a quick one-off python script to do just that:

-------------------------- BEGIN magicslide.py
# !/usr/bin/env python

"""

%s <filename>
<filename> will be checked at each  offset to see what the magic offset
database from the "file" command's database thinks it is.

Entries that return 'data' will be filtered because they are boring.

"""

import magic

import os

import sys

def usage():

    sys.stdout.write( __doc__ % os.path.basename(sys.argv[0]))

    sys.exit(0)

def analyze(ms,buffer):

    return ms.buffer(buffer)

def output(offset,s):

    sys.stdout.write("%08x:%s\n" % (offset,s) )

try:

    filename = sys.argv[1]

except:

    usage()

try:

    f = open(filename)

except:

    sys.stderr.write("could not open %s\n" % filename)

    sys.exit(1)

filedata = f.read()

totallen = len(filedata)

buffsize = 4096 # a nice big chunk of file

# load the magic db

ms = magic.open(magic.MAGIC_NONE)

ms.load()

for offset in range(0,totallen):

    end_offset = min(offset+buffsize+1,totallen)

    kind = analyze ( ms, filedata[offset:end_offset] )

    if kind != 'data':

        output( offset, kind )
--------------------------------------- END magicslide.py

Sample output looks like:

0001047c:Hitachi SH big-endian COFF executable, not stripped

00010493:PCX ver. 2.5 image data

000104a8:MIPSEB MIPS-III ECOFF executable not stripped - version 255.26

000104b2:\012- 8086 relocatable (Microsoft)

000104b8:PCX ver. 2.5 image data

000104bd:MPEG ADTS, layer I, v1,  32 kBits, 32 kHz, Monaural

000104c1:MPEG ADTS, layer I, v1, 448 kBits, 32 kHz, Stereo

000104c8:DBase 3 data file

000104cc:LANalyzer capture file

000104e0:PCX ver. 2.5 image data

000104e8:shell archive or script for antique kernel text

000104ef:PCX ver. 2.5 image data

000104f6:MPEG-4 LOAS

00010508:AmigaOS bitmap font

0001050c:PCX ver. 2.5 image data

00010514:shell archive or script for antique kernel text

0001051c:MIPSEB MIPS-III ECOFF executable not stripped - version 0.10

00010522:MPEG-4 LOAS

00010530:Hitachi SH big-endian COFF executable, stripped

00010538:DBase 3 data file

0001053c:PCX ver. 2.5 image data

00010544:shell archive or script for antique kernel text

00010549:MPEG ADTS, layer I, v1,  32 kBits, 32 kHz, Stereo

00010560:DBase 3 data file

Well, it’s still pretty messy and the data may be wrong, but it’s more than we had to go on before for our analysis of this unknown file type. There are obvious false positives here, but things like images such as JPGs, PNGs, etc. can probably be readily identified in the file of interest.

# aa

2 comments

I just got back from the IT Security World Conference & Expo 2008.  This was the first time I’ve attended this conference. The speaker line up looked good. I wasn’t there to see the speakers though; I was an exhibitor working a phishme booth.

I’ve spoken at DefCon, BlackHat, Shmoocon, etc…. but at this conference, I wore my exhibitor badge, which might as well have read “leper”.  Hah, not that I can blame the attendees for treating me like a leper, after all, I was just another exhibitor in the gauntlet they had to run in order to get to the drinks and snacks. 

When you brave the booth gauntlet, you’re bombarded by shiny people. Appliance after appliance, magic boxes that make all your IT security problems go away.

My booth was at the end of the gauntlet. It was entertaining to watch attendees pickup my swag without missing a step, only to read the banner that says “Phish your employees” pause, double back, and curiously ask me  “what is this?”  Most would chuckle after figuring out exactly what phishme.com does. Eyes popped out of heads of the ones that actually saw the demo.

There was something about the conference and expo that REALLY bothered me……

 

 

 

 

The sad thing was these Internet terminals were in heavy use throughout the conference. Every time I walked by them people were in their email. 

 

-higB

 

1 comment

Better late than never right?

Since we basically missed all of Blackhat except Schmoilito’s talk this year (hey, pool security is important too), I’ve made a list of the best Defcon talks I heard this year. To sum it up: Cable Modems, Wifi, NMAP, and Mati Aharoni.

Both Guy Martin’s and Blake Self’s talks on cable modems were eye openers. You could have probably guessed people were writing their own firmware for cable modems to unlock their full potential, but it was interesting to get the background on it and an overview of DOCSIS. Mr Martain’s presentation then showed what mass pwnage really looks like by sniffing a network at cable modem speeds using an inexpensive DVB-C card. It also wasn’t over looked that his “packet-o-matic” tool had one of the best user interfaces for any home grown tool we’ve seen in a long time. A web interface with smooth AJAX requests. Sure, GUI’s are for script kiddies, but good GUI’s are like the same reason the chicken wings at Hooters taste so much better.

In the WiFi world, Rick Farina and Thomas d’Otreppe talk was interesting especially in regards to unlocking the 4920-6100 MHz range. I’m wondering if we’ll ever see this in an assessment, but the idea of running your own home wireless network outside the range of normal prying eyes is very intriguing. The ath5k frequency patch appears to now be online. Still looking for a Wii patch to support this…

While I was in the cable modem talks, the network guys hit Fyodor NMAP talk. From the twitter comments, he rocked it with some cool new updates to NMAP and a Netcat replacement tool. I thought I had too many beers when they mentioned the Netcat replacement, but it sounds like Fyodor and team’s Ncat has a lot going for it. SSL support, port redirection, built in proxy and access control support. Definitely worth checking out.

The last thing on my highlight reel was Mati Aharoni’s “From bug to 0day” talk. Mati showed he must make one hell of a teacher in the Backtrack classes. He basically told the story of what he needed to go through to find an 0day in a client’s project. It was a great walk through of both the technical and thought process and not just a walk through of slides (don’t bother with the slides on this one, you needed to see his screen and hear him). I think Mati got everyone in the room sharing his tension and completely wrapped up in the adventure. I wanted a box of popcorn and a squeal when he was done.

DefCon badges were once again, awesome.

… I completely missed Sunday’s talks. I heard good things about Carric’s Pen-Testing presentation. I plan to catch that on the DVD.

-b3nn

No comments