pw2kindle: a perl projet meant to experiment

12 days ago

Intro

I love my kindle. With it, I’ve been reading even more than before. Especially during commuting. I read books at first but then discovered instapaper which allows you to push articles in a ‘to read’ list which then gets converted into a magazine format and wirelessly delivered to your kindle for free! This is so convenient that it went to the point where I’ve mostly stopped reading in front of the computer.

I love perl! I love reading about perl! As a “born-again” perl developer I’ve stumbled on perlsphere and have been browsing it more or less thoroughly since 2-3 years. I say more or less thoroughly due to the fact that I have to remember to come back to it from time to time. Then, lately, Perl Weekly arrived into our lives, and I thought: “If I could read that on my kindle it would be awesome!”.

Enter pw2kindle.

First, I must say that my next from-scratch perl project had to incorporate new and fancy things as a test bed to simplify stuff at work. pw2kindle is that next project. What does this means? This software is completely overkill and out of proportion for such a simple task.

Here’s a quick rundown of the stuff I tried and the stuff I learned:

Command line application framework

I wanted to play with CLI framework. First and foremost because a simple perl script isn’t testable until you factor it out into a module. But mainly because, at work, we have probably one of the worst case of a command-line application that I would like to split into chunks and refactor (but never got the time to… you know, the reality of a small-business doing open source…).

So I wanted to experiment with command line applications frameworks. After some research:

Were the candidates. In the end it came down to MooseX::App::Cmd versus CLI::Framework. I chose MooseX::App::Cmd because of Moose. I have always wanted to try Moose out on a ‘real’ problem (instead of doing tutorials). Although I have to give a mention for CLI::Framework (CLIF) which seems definitely the more powerful one and then one I would’ve gone with if it was not for Moose.

Web::Query

Another great thing I played with is Web::Query. To quote the author’s description:

Web::Query is a yet another scraping framework, have a jQuery like interaface.

Look at the kind of terseness you can achieve with it. The following adds $article objects to the @articles array that come from all <p class=entry><a class=...>title</a><p>description</p></p> in the given URL:

my @articles;
wq("perlweekly.com/archive/$issue.html")
    ->find('p.entry>a')
    ->each(sub {
        my $i = shift;
        my $article = Pw2Kindle::Model::Article->new(
            title => $_->text,
            url => $_->attr('href'),
        );  

        printf("%d) %s\n", $i+1, $article->toString()) if $self->dryrun();
        push @articles, $article
});

The beauty is in the find() with it’s CSS style selector (borrowed from jQuery I guess?) and then the ability to run an each with a closure on it. Very neat!

Problems along the way

The frame that App::Cmd provides needs some effort learning but I also see a lot of benefit in the boiler plate removal. It’s doing all the splitting for you, enforcing some structure in the package naming. Also, it recommends using autouse so large command-line applications probably benefit from it performance wise.

That said, I spent some time in the debugger tracing problems blaming issues on Moose although it turned out that it was a user problem.. To my defense, the documentation doesn’t covers unit testing.. So I spent some time thinking it was a MooseX::App::Cmd to App::Cmd mismatch problem instead of me misusing the framework trying to test.. In any case, I traced it and fixed it, it was my fault using new() instead of prepare().

What is pw2kindle

Right now, being mostly focused on getting Perl Weekly on my kindle quickly, here’s what the thing does:

What should be pw2kindle

Ideally I would like to do the ‘instapaper’ text (and relevant image) grinding myself but I have another project that needs attention right now (shameless plug: Hacker Jeopardy @ HackUS 2012) so I won’t touch it for little while.

Concluding

The Moose learning was awesome! I love that stuff although I barely used it. I enjoyed the structure and splitting provided by the CLI framework. I’m definitely re-using one of these on any relatively large command line application. I hope I will have the chance to make this one grow.

Now, I can’t wait read Perl Weekly on my kindle.

Feel free to fork pw2kindle! Check out the TODO if you are interested to contribute.

— Olivier Bilodeau

Comment

spacer

Article on PacketFence

106 days ago

My article on PacketFence was published today in the last issue of (IN)SECURE Magazine.

It’s quite technical and it is all new material.

Go ahead, read it and give me feedback!

Direct link to issue 32.

Cheers!

— Olivier Bilodeau

Comment

spacer

Here's what really happened at RSA

191 days ago

I originally posted this late at night to a local security group. Then I thought it was funny and worth translating and sharing on my blog.

Note: Nothing in here is based on actual evidence. This is just an exercise for fun.

As I write this, everyone is getting their RSA tokens replaced. They think they are getting new ones because their old ones were compromised but what if things are not exactly as they look?

Here’s what happened: RSA was not keeping their seeds for the tokens. Because, after all, they didn’t need the seeds. Once the token’s seed database was delivered to their customers, they diligently deleted the seeds they had. It was the most secure thing they could do.

Sometime later, the NSA1 knocked at their door saying: “We would like to have all the seeds for your tokens in exchange for a sh*tload of money. You see our old usual win32 0-days aren’t cutting it anymore.. With the seeds we would be in way better shape to attack the people who are a threat to our nation.”

RSA replied: No but not because we don’t want to, because we can’t.. We never kept them, the most secure thing we could do for our customers was not to keep them.

Then it got into an RSA sales guy and/or C-level manager’s ear. They mandated their best technical people to it and they came up with a plan:

“Let’s do a big ‘we got owned’ fiasco story and tell all of our customers that we need to give them new tokens because the old ones aren’t safe anymore. But this time, we keep the seeds!”

NSA is happy. RSA is happy. USA is happy.

And of course, this is not what really happened.

1 RSA, NSA only one letter difference.. ;)

— Olivier Bilodeau

Comment

spacer

Tomdroid 0.5.0

194 days ago

spacer I released Tomdroid 0.5.0 earlier this week and I’ve heard no big complains that it ruined someone’s life. Here’s the announcement.

I’ll spare you the details but as you can see, it sports a new icon.

And here are some screenshots: spacer spacer

Enjoy!

— Olivier Bilodeau

Comment

spacer

Hackus: three days of intense hacking challenges

333 days ago

Just a quick post to say that I had a blast at the last Hackus. The whole competition was organized top-notch and the challenges gave us more to do than we had time for.

spacer First, I’ve got to say that I always preferred the CTFs where we are all together than the ones that we do over the Internet so, Hackus being one of the former category, its a plus for them.

Then, the challenges, there was: a forensic challenge who quickly turned out into a twisted defensive / offensive CTF, Java reverse-engineering, Web exploitation, service exploitation oriented CTF, Web Application Firewall Evasion challenges, a very interesting and twisted social-engineering challenge with live actors and unexpected illegitimate access to hardware, cryptography, a handful of networking challenges with an unexpected IPv6 appearance (and we are sooo not ready!), general puzzle-solving, geek/hacker/cracker knowledge, hacker jeopardy, steganography, parties with DJs and more that I forget.

The organizers really outdid themselves: There was a lot of stuff covered, it was tremendously hard (keep in mind its all volunteer work!) and their infrastructure hosted all of this without even a wink. An all-across-the-board succes!

The networking challenges were especially welcomed from me since they are quite hard to set-up infrastructure-wise, never seen on CTFs over the Internet and are, in my opinion, quite important and often overlooked.

spacer Finally, I guess the fact that we won probably also weighted in the fact that I enjoyed the event. That said, the competition has been very fierce until the end. Our team, Amish Security, had the chance of combining a lot of diverging abilities and talents into a cohesive team. Let me name drop here: Guillaume Germain, Gabriel Tremblay, François Proulx, Laurent Desaulniers, Pierre-Marc Bureau, Joan Calvet, Benjamin Vanheuverzwijn. All awesome, hard-working and talented people!

I also organized the Hacker Jeopardy portion of the event, which will have its own post.

I said short post at the beginning, I know, sorry about that..

Pictures by Rémi Menegon used in accordance to the CC license

— Olivier Bilodeau

Comment

spacer

Older

Related searches:
kindle framework because article weekly
gipoco.com is neither affiliated with the authors of this page nor responsible for its contents. This is a safe-cache copy of the original web site.