«

»

Dec
18

RuCTFe rocks, iCTF is fine, rwthCTF are assholes

The traditional Winter Attack-Defense Spree is over, and what can I say…

pic

RuCTFe 2013 rocks

First, let’s begin with RuCTFe 2013.
RuCTFe is an annual competition hosted by Hackerdom. 2013 was its 5th year.

RuCTFe is perhaps the most Classic Attack-Defence out there. Since its first year, the rules stay pretty much the same: there is a vulnbox with a pack of services, you gain points for attacks, you gain points for keeping your services up and unpwned.
This year Hackerdom experimented with scoring, making it Attack * SLA, which raised SLA valuability a whole order of magnitude, but well, it worked fine for us :)

The thing to highlight is the flawless organization and zero fuckup level this year. That’s definitely something hard to achieve for an A&D. Nice job!

iCTF 2013 is fine

Going backwards on the timeline, iCTF 2013.
iCTF is one of the most aged and respectful international Attack-Defence CTFs and has been hosted by UCSB for the 11th year in a row.

iCTF is the most Stochastic Attack-Defence competition. One can never guess what to expect from a next iCTF. Every year the organizers at UCSB seem to roll huge dice with CTF legend ideas and infrastructure layout. This year, the infrastructure die rolled out the same thing as previous year. Rare case!

Anyway, iCTF is fun each year, expecting the teams to understand game mechanics first instead of plain hacking.
My opinion: the mechanics turned out to be too random this year. Seems like only the last 1/4 of the game is where you actually get scores (the point reset thingy), and the “turn off opponent’s exploits” is imba since there is no way to escape it.

rwthCTF 2013 is nice, organizers are utter assholes

And finally, rwthCTF 2013. It is hosted by European RWTH Aachen University, which also hosted Cipher CTF back in the days.

This year’s rwthCTF was the most Assholish Attack-Defence competition. Challenges were fine. The idea with separate ARM service was nice, although it has immediately cut down the number of teams to 64.

The phailure was organization. Imagine a situation like this:

Your team is having a good luck in a CTF, and holds 1st place pretty confidently for 8 hours straight, out of 10h total playtime. 2.5h before the end, you exploit a new vuln in the service, and – bingo! – 90% of your opponents are still vulnerable. Woot! pretty awesome scenario, right? You’ve been lucky enough to trade your effort for a hard exploit that can give you many flags and increase your lead :)

Using the vuln you found, you begin stealing flags from all other teams, and also start removing flags from their boxes for them to lose defence points. Everything goes well, you get a 50 “Congratulations, you have scored a point!” scorebot messages every few minutes, nice! Then after a few minutes you start to notice that your score isn’t growing that fast :-O

Actually, it seems like none of the flags that you’ve stolen from that service get counted. You PM an organizer asking to look into the issue. “Hm, the flags are in the database – they should have been counted” is his answer. OK, you wait, maybe it’s just not displaying the score properly because of caching or something… Nothing. You continue to bug the org about this thing, and – at last! – after almost an hour he responds “Oh apparently the flags weren’t counted because you delete them from opponents’ boxes. Now we changed the scoring logic so flags should be counted fine”

Oh thank you, – you think, – but we’ve had this issue for an hour already and have lost a ton of points, can we get those re-counted since you have them in your DB? – “Meh, well… maybe we’ll talk about that later…”
WTF?! As you can tell from the numbers, your team would have twice as much points at that moment, if your flags were being counted properly.

So anyway, while the org is doing nothing and the game is ending its course, two other teams overcome you, and you’re on 3rd. The game ends. You remind the org about score recalculation. He starts debating with the CTF dev team about how much points your team has lost, it turns out if orgs do a recalc, you win the ctf – 1st place instead of 3rd. At the same time, he tells you that you’re the wrong party here: you shouldn’t have removed the flags from opponents’ boxes, irregardless there is no such rule in any CTF. Also it’s 5am at your place and you’ve been pwning for 10 hours straight, and they’ve been supporting the infrastructure for 10 hours straight, so everyone’s irritated as hell.

Finally, guys from 1st and 2nd place teams start whining and the orgs decide to do nothing. You’re on 3rd while having pwnd enough to be 1st.

Surrealistic? Well, that’s what was actually happening.

The day after, after having some sleep and relieving from CTF hangover, I have another discussion with that org, about why they decided to do that – and instead of even admitting he’s wrong, he tells me like, “that was the rule dude, you shouldn’t have removed the flags” and claims that i’m the asshole here for bringing it up again. :-(

Gathering it all together:

  • The game has a rule which isn’t in the rule set – teams have no way to find it out
  • In fact, in the rule set there was this (like in any other A&D CTF): “However, if other teams manage to delete your flags, you will not receive points” – in my opinion meaning exactly “if you want opponents to lose defence points, delete their flags”
  • The scorebot responds “Congratulations, you have scored a point!” when you post the flag – like you would expect on any CTF – but doesn’t actually add any points
  • When you ask the orgs about why the flags aren’t counting, it takes them a fucking hour to figure out this rule
  • We lose 1/3 of our points because of the rule, and the only thing orgs are doing is saying “oops, sorry for not telling you about that rule”
  • Orgs refuse to publish the game logs for everyone to see

For the historical purposes, here are the text-fighting logs

Don’t know whose decision it is, but in my opinion that’s not how any competitions should be organized.

/!\ Asshole alert /!\

14 comments

  1. rep says:

    So pals, even though you’re being unprofessional with your post and language, here is a comment on the facts. But before I do I would like to remind everyone that hosting a CTF competition is a lot of work and has pretty much no other purpose than fun and learning experience on both sides. We definitely had fun organizing and learned a lot and we got the feedback that the same is true for almost all teams participating.

    Regarding the fuckup / rwthCTF situation:
    YES, WE SCREWED UP! We know that a big mistake was made and we apologized several times (several organizers did) for NOT IMMEDIATELY CLARIFYING the flag overwrite game logic. In the heat of the moment we were unable to give you a direct and clear answer. ALSO WE SCREWED UP that this aspect of the scoring / rules WAS NOT LISTED on the website / scoreboard.

    SO YES, we fucked up. And we’re really sorry.

    Now that being said – as other teams noticed the same situation, they STOPPED overwriting flags and by that adapted to what was happening and continued playing (e.g. Eindbazen). Which is what every hacker probably would do in this situation (except you guys obviously).

    TO CLARIFY WHY SCORING WORKS THIS WAY READ ON:

    In online classic CTF competitions that include a vulnerable image which every team has to host, there are a number of problems / pitfalls to avoid from a network / rules / scoring perspective. One of these pitfalls, especially with growing participation and thus bigger CTFs is network traffic / packet rates that are going on with >100 teams exploiting the shit out of each other. Things are going to become slow – especially the CTF services themselves take quite a beating with all these incoming connections.

    Now, if you do not somehow discourage overwriting (or DoS for that matter) through game design / scoring, people will go crazy and there will be a fight of who comes first, rather than who is best. If you allow OVERWRITING of flags in ANY of these CTFs of this type, it becomes a race. Teams will launch exploits every few seconds, as they want to steal and delete the flag.

    This is CLEARLY not something you would want. Of course IN GENERAL it would be nice to overwrite flags and thus limit teams from scoring – which is why DEFCON has this feature – and they are able to make it happen because they have a kernel module in place that COUNTS the overwrite, but DOES NOT ACTUALLY overwrite the flag.

    This is why rwthCTF ALWAYS HAD this particular scoring scheme – a flag will only be counted in the pool of possible flags, if we were able to retrieve it from the service again. This is the decision we made for limiting DoS and overwriting.

    AGAIN guys, WE SCREWED UP on communication and rule definitions. YES IT SUCKED that this happened. And we’re still sorry. However there was no possibility to to re-scoring without being unfair to the other teams and thus we had no choice as to keep things as they are and distribute the prize money in a little bit flatter curve than planned.

    So honestly, stop whining about it and let’s sit down with a beer and laugh about this mistake. Also of course everyone is free to organize a better CTF, putting in man-months of work.

    Cheers guys,
    -mark

    1. vos says:

      Hi mark, thanks for commenting.

      First of all, explaining why I go for “publicly attacking” rwthCTF orgas. It’s been over a month now and you guys didn’t publish any info on that issue (neither your sorries, nor game logs), despite we asked you to – so i wanted to post something to explain people why I’m calling RWTH organizers assholes (or the one that I had the IRC discussion with personally).

      Second, I’d like to clarify what I see as wrong in RWTH orga’s actions. Fucking up is alright, no-one is safe from doing stuff wrong. The worst part is, instead of fixing the fucked up stuff you have decided to limit yourself to just saying “oops, sorry for fucking up the rules” and doing nothing.

      Imagine you’ve got a nice position in some respectful company. In the end of the month you go to get your salary.
      — Let me check… — says the accountant — You’re not eligible for your wages: our records show you’ve been wearing your underwear to work.
      — Wearing underwear?!
      — Yeah, we have this unspoken rule in our internal codex, anyone who chooses to enroll for any money shouldn’t wear pants to work.
      — …
      — Sorry for not telling you about the rule beforehand, trololo.

      There is this universal excuse: “re-scoring would affect other teams“. Would it affect their score? No — at least not as hard as ours. Would it affect their positions? Yes — but it was your decision to postpone the recalc to the furthest possible point in time, when it definitely will influence the final team ranking.

      distribute the prize money in a little bit flatter curve than planned

      — BTW, we also include 4th place in prizes, trolol.

      Funny bit: when in the end they announced that they “flatten prize distribution curve” I was like — …are you kidding me, there were prizes and not just playing for own fun, and orgas decided to leave the final ranks as they are?! Neither of us (MSLC) knew till the end that winners will get any rewards, so it was even more frustrating to know that “oops, you’re now 3rd guys”

      The motherfuckin’ saga continues

  2. rep says:

    We haven’t gotten to making a blog post yet, pretty good reason to flame us this hard.

    Mistakes were made – however there weren’t any viable fixes. Again, the explanation was given at the end of the CTF and in my comment up there as well. Not really hard to understand – it would have been unfair to recalculate scores (if it was even possible to implement correctly in that short timeframe). It does not matter if you think you’re the only one affected or that it would only have been “a little unfair” to others. It wasn’t viable, that’s it.

    Your story about the salary is so far-fetched and unrelated, it really is almost funny.

    We flattened the prize curve to 4th place as they were super close to you guys in score – so maybe with rules being clear or with a recalc you would have been on 4th and they on 3rd? Who knows? So again, trying to be fair. Not hard to understand.

    (final scoreboard http://ctf.itsec.rwth-aachen.de/final_scoreboard.html)

    There was no “oops now you’re 3rd” – you were 3rd – nobody changed anything in the scores.

    I think you did an awesome job exploiting stuff – you’re clearly a good CTF team. It was very unfortunate what happened and we’re sorry for it. However I’m amazed by your ignorance of those apologies and that you do not accept the offer of beer by a german. That’s really rude.

  3. qll says:

    tl;dr: The CTFs you didn’t win were stupid? :P

    @ruCTF: Good orga, good challenges, but strange things happen when you tell the orgas that you have access to another russian team’s server ;)

    @iCTF: I don’t even… “too complicated” is probably a very nice way to put it

    @rwthCTF Whatever. Should we open a donations account for the poor MSLC players so that you’ll get your prize money? :)

    1. Zaza says:

      For the first, considering that MSLC team won as #1 _MANY_ CTFs, team is not poor, so stick your donation suggestion up your *ear*. You get the point. And rwth argument has never been about the money.
      From what it looks like, rwthCTF simply screwed up MSLC by making unfair judgement.

      If they acknowledge that they have screwed up and scores of leading(!) team were affected because of those screw-ups, they should take action and recalculate in a fair way (REALISTIC) way. But what organizers are doing? Right…all you see is: ooooh, we’re sorry, we screwed up, sorry, blablabla…. Not good enough. Not a fair way to treat top level teams like that. It doesn’t matter if it’s MSLC, PPP, Eindbazen, Faust, Enoflag or any other world famous winning team – the point is simple> If orgs have screwed up, they should deal with it, instead of simply apologize — THAT’S not good enough.

      Bravo to vos – comparison about the salary is a bull-eye, at least if I was playing that CTF and got in that mess as MSLC team did, I would be definitely pissed off, and for the second – definitely expect corrective actions from organizers, and not just words and some cheap beer.

      What happened with you, rwth guys? Awesome challenges, but to fuck the leading team like that – shame on you. In other words, you deserved all the shit which was published here. And it’s never too late to deal with it properly. (It’s not about the money, but about fair decisions).

  4. vos says:

    mark, as far as you are incorrectly interpreting my words, I’m obligated to correct you.

    It does not matter if you think you’re the only one affected or that it would only have been “a little unfair” to others.

    A fair recalc won’t be “a little unfair” to others, it would only raise their scores, not lower them. I’m just saying it would probably raise ours more than others’. I’m limited to empty assumptions here, since you’re the ones holding the DB – you have the info to check that.

    Your story about the salary is so far-fetched and unrelated, it really is almost funny.

    But maybe it illustrates the principle of an unknown hidden game-changing rule pretty clearly.

    However I’m amazed by your ignorance of those apologies and that you do not accept the offer of beer by a german.

    ;) And you’re offering a russian something that is not vodka.
    As for the apologies, we have seen tons of your apologies for not telling anyone about the rule (and actually having an opposite rule in the ruleset), and we appreciate them but honestly the thing that I personally would be really happy to hear is that you understand that in that situation you were wrong not in the rules but in doing nothing other than apologizing for the bad rules

     
    qll, tl;dr: ruCTFe was nice, iCTF was a bit stupid (too random), rwthCTF was nice and orgas were stupid.

    strange things happen when you tell the orgas that you have access to another russian team’s server ;)

    Ohh, so the CTFs you didn’t win were stupid? :P JK, nice challenging competition on ruCTFe with you guys.

    UPD: BTW, since you’re joking about donation account, it deserves a separate mention that we didn’t even know there were prizes till the CTF was over. It’s not about the money, it’s about sending a message.

  5. vos says:

    Crosslinking to RWTH’s blog post on the same issue: http://oldeurope.github.io/2013/12/19/Scoring-in-rwthCTF-2013/

  6. rep says:

    I promised I’d be done with the discussion after my last post, but I just can’t resist… rwthCTF website, weeks before the event:

    Thanks to generous donations by our sponsors, we are able to offer cash prizes for the winning teams this year.

    1. vos says:

      Yeah, we were just careless enough to read over it, not saying you didn’t mention the prizes. Entirely our miss here.
      Just saying it didn’t affect how we played the CTF or how frustrating it was afterwards.

  7. kyprizel says:

    Every CTF organizers should understand that any of their fuckup multiplies on every teams playing the CTF time. So, yes – it’s hard to be an CTF organizers and it’s much harder to be the organizer of good CTF event.

    All we play CTFs not for winning and of course not for prizes or money. We do it for fun. Everyone gets their fun – players, playing the CTF, organizers – doing the event.

    Personally I don’t like the idea of calling someone an asshole.

  8. tr0ll says:

    This post will talk about how much Leet More Smoked Chicken are a bunch of whiny bitches who should be banned from the internet. Forever.

    INTRODUCTION

    rwthCTF2013 took place November 9, 2013. Unlike the last two installments, a team who consistently makes DEF CON finals every year gets all butthurt about math and makes an angry post on their blog.

    GAME THEORY (NOT REALLY)

    tl;dr

    THE MISTAKES / SCREW-UP

    First: Having such a bunch of alcoholic whiny Russian elitist nerds participate in the first place. Next year this problem will be fixed.

    Second: Cash prizes.

    Third: Lack of a complaint department. In the future, please send all CTF related complaints to tr0ll@legitass.net

    CONCLUSION

    The mistakes that we were guilty of for rwthCTF 2013 were mostly due to climate related issues in 1941. Next time you will not be so lucky.

    1. Wizard of Oz says:

      2 tr0ll: Wtf?
      You’re stuck in 1941….ouch, no wonder your brain is a size of a nut.
      Bbbb…ullshitttt detected! *caugh* *caugh*

  9. psifertex says:

    Running CTFs is hard, it’s important to correct what you can, but sometimes stuff gets screwed up, and organizers have to apologize, and players have to roll with it.

    Did you (MSLC) enjoy good challenges and have fun along the way? Great! It sucks that the score might not be representative of the work you did, but I agree with kyprizel. Calling the organizers assholes because you disagree with their judgement in a tough spot is not helpful.

    Every CTF has its ups and downs (some worse than most, to be fair). Take the good for what you can, see if you can offer constructive criticism (re: not name-calling). I’ve got my pet peeves from certain CTFs, but I’d never say anything ill about the organizers, just that I disagree with either their priorities, or some choices.

    OldEurope apologized for the obvious oversight, and having gotten into that situation in the first place, there’s pretty much no optimal outcome. You can argue whether what they did was the right call or not, but let’s at least try to be constructive about it.

    1. Wizard of Oz says:

      psifertex is right about calling names. However, would you feel to be fairly treated if you’re crossing finish line first, and then someone says – sorry, you’re not #1, you’re not even #2, and ooh..sorry again, you’re not even #3, you’re what we feel is right – let it be number 4 because we feel like it.

      Fair?
      I don’t think so, no matter how you look at it.

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Current day month ye@r *