Jordan Sissel
hacker
Combine something simple like Firefox and Xvfb (X Virtual Frame Buffer), and you've got a simple way to run Firefox without a visible display.
Let's start Xvfb:
startx -- `which Xvfb` :1 -screen 0 1024x768x24 # Or with Xvnc (also headless) startx -- `which Xvnc` :1 -geometry 1024x768x24 # Or with Xephyr (nested X server, requires X) startx -- `which Xephyr` :1 -screen 1024x768x24This starts Xvfb running on :1 with a screen size of 1024x768 and 24bits/pixel color depth. Now, let's run firefox:
DISPLAY=:1 firefox # Or, if you run csh or tcsh env DISPLAY=:1 firefoxSeems simple enough. What now? We want to tell firefox to go to google.com, perhaps.
DISPLAY=:1 firefox-remote www.google.com/Now, let's take a screenshot (requires ImageMagick's import command):
DISPLAY=:1 import -window root googledotcom.pngLets see what that looks like: googledotcom.png
While this isn't complicated, we could very easily automate lots of magic using something like the Selenium extension, all without requiring the use of a visual display (Monitor). Hopefully I'll find time to work on something cool using this soon.
Problems with screen scraping and other website interaction automation is that it almost always needs to be done without a browser. For instance, all of my screen scraping adventures have been using Perl. Browsers already know how to speak to the web, so why reinvent the wheel?
Firefox has lots of javascript-magic extensions such as greasemonkey and Selenium to let you execute browser-side javascript and activity automatically. Combine these together with Xvfb, and you can automate lots of things behind the scenes.
Tie this back to unit tests. Instead of simply displaying results of unit tests, have the page also report the results to a cgi script on the webserver. This will let you automatically test websites using a web browser and have it automatically report the results back to a server.
My "vim scripting"-fu is not strong, so there's probably a cleaner/fancier way to do this. Anyhoo, you'll need the following in your .vimrc:
" PyBlosxom stuff
augroup pyblosxom
autocmd BufReadPost /home/jls/public_html/entries/*/*.txt call Pyblosxom_checkdate()
autocmd BufNewFile /home/jls/public_html/entries/*/*.txt call Pyblosxom_putdate()
autocmd BufWritePost /home/jls/public_html/entries/*/*.txt call Pyblosxom_fudgedate()
augroup end
function Pyblosxom_checkdate()
" Look in the file for '#mdate foo' metadata
normal 1G
let dateline = search("^#mdate")
" If not found, append the mdate of the file to line 2
if dateline < 1
let dateline = 1
let date = system("stat -f '#mdate %Sm' " . expand("%"))
" Add the date to the file on line 1
1put=date
endif
endfunction
function Pyblosxom_putdate()
let date=strftime("#mdate %b %e %H:%M:%S %Y")
1put=date
goto 1
endfunction
function Pyblosxom_fudgedate()
let l=search("^#mdate")
let l=strpart(getline(l), 7)
let cmd="date -j -f '%b %e %H:%M:%S %Y' '" . l . "' +%y%m%d%H%M"
let touchtime=system(cmd)
let touchcmd="touch -t '" . strpart(touchtime,0,strlen(touchtime)-1) . "' '" . expand("%") . "'"
call system(touchcmd)
e
endfunction
nfsstat | sed -ne '/Version 3/,/^$/p'sed++
When I was bored (at 4 am, no-less), I kept trying to parse this information out using some crazy tricks with 'x' (swap pattern/hold) and other stuff, but I forgot the fact that regexps are valid addresses. So, we can print anything between 'Version 3' and blank lines, anywhere in our output.
The next thing I want to try with this is to automagically parse nfsstat output into a format that is more machine readable, this will probably be using awk or perl, seeing as how doing it with sed may hurt my brain a bit. Furthermore, trying to read the sed that did said operations would be somewhat intense ;)
The output looks something like this, on Solaris 9:
Version 3: (535958 calls) null getattr setattr lookup access readlink 0 0% 242223 45% 20606 3% 52504 9% 20025 3% 41 0% read write create mkdir symlink mknod 14138 2% 146618 27% 5525 1% 145 0% 337 0% 0 0% remove rmdir rename link readdir readdirplus 6279 1% 7 0% 1539 0% 1518 0% 1606 0% 6587 1%Parsing this would mean generating a tree-like dictionary. In perl, it may look like:
%foo = (
'Version 3' => {
null => 0,
getattr => 242223,
setattr => 20606,
lookup => 52504,
# .... etc ...
}
)
Should be simple enough, we'll see what happens next time I get bored.