Oddly Shaped Pegs

An inquiry into the Nature and Causes of Stuff

Systematization of Knowledge track at Oakland

with 5 comments

I am on the PC for “Oakland” this year (a.k.a. the IEEE Symposium on Security and Privacy).

I have been on the PC of a few conferences in areas outside my immediate expertise and so far I’ve enjoyed the experience. Usually, I am asked to join because they need someone to help them reject carefully review the few crypto/privacy papers that get submitted. Along the way, I get to learn about a different area of research, and about the taste in problems that is prevalent in the other community. Oakland is different because it is nominally about  my area, but the author community is essentially disjoint from the STOC/FOCS/Crypto/Eurocrypt crowd that I hang out with; consequently, the focus of the submissions is very different from that of the papers I am used to reading. I promise to share any (constructive…) comments I have on my experience.

Anyway, my main point: this year’s Oakland will include a new “systematization of knowledge” track. The call for papers says it all:

“The goal of this [track] is to encourage work that evaluates, systematizes, and contextualizes existing knowledge. These papers will provide a high value to our community but would otherwise not be accepted because they lack novel research contributions. Suitable papers include survey papers that provide useful perspectives on major research areas, papers that support or challenge long-held beliefs with compelling evidence, or papers that provide an extensive and realistic evaluation of competing approaches to solving specific problems. … [Submissions] will be reviewed by the full PC and held to the same standards as traditional research papers, except instead of emphasizing novel research contributions the emphasis will be on value to the community. Accepted papers will be presented at the symposium and included in the proceedings.”

Well-written regular papers already include a “systematization of knowledge” component in their related work sections: the obligation to summarize related papers often results in a clean, concise presentation of their high-level ideas. Unfortunately, the quality of the related work section rarely makes or breaks a conference submission, so mileage varies; hence the need for a separate track.

A few questions jump to mind:

  1. If this “systemization” track becomes standard, how will job candidates be viewed if their publication lists contain many such systemization papers? A successful textbook can dramatically increase a researcher’s profile; is the same true of survey papers?
  2. What areas of crypto/security/privacy are in direst need of “systemization”? Here a few suggestions for Oakland-appropriate topics:
    • definitions of security for anonymous communication systems (e.g. Tor)
    • techniques for de-anonymization of sanitized data (hopefully tying together papers published at Oakland, KDD, SIGMOD, VLDB, ICDE, etc)
    • Notions of “computation with encrypted data”: homomorphic encryption,predicate encryption, deterministic encryption, order-preserving encryption, etc.
  3. Assuming the Oakland systemization track is a success, what other conferences would benefit from adding such a track?

Written by adamdsmith

July 9, 2009 at 12:21 pm

5 Responses

Subscribe to comments with RSS.

  1. I think this is a great idea. I have one suggestion — if people are working on survey papers, please do so “in the open.” If you have a blog, announce it there, otherwise send early drafts around to colleagues, etc. Since no new research is involved here, duplication of effort would be really unfortunate.


    July 10, 2009 at 1:11 am

  2. Congratulations with the PECASE! And great that you started to update the blog. Some good questions/thoughts there. You should advertise it more at CRYPTO!
    – Yevgeniy


    July 10, 2009 at 12:09 pm

  3. Interesting idea, though somehow a position paper seems more natural to have at a conference than a survey.

    How will they deal with the inherent conflict between a good survey and conference page limits?


    July 10, 2009 at 9:49 pm

  4. This is the first year that Oakland is trying this track, so I expect there will be a fair amount of discussion among the PC about what is appropriate.

    > Interesting idea, though somehow a position paper seems more natural to have at a conference than a survey.

    I think that some position papers could, in fact, fit the call, as long as they are technically oriented (“papers that support or challenge long-held beliefs with compelling evidence”). Jon, Do you have any specific examples in mind of already-published positions papers that would have been good at a conference?

    > How will they deal with the inherent conflict between a good survey and conference page limits?

    TBD. I think that for theoretical topics this will definitely be a problem. I will be interested to see how people choose to use the space.

    I’ll add one of my own concerns: surveys are often written by an author about their own work, or topics very close to it. Such a submission might run into (justified) skepticism on the committee about how broad the community’s interest is. Should such self-surveys be discouraged? Is it double-dipping?


    July 10, 2009 at 10:00 pm

  5. I don’t think we should discourage self-surveys. A survey serves a different purpose than a research paper, and the persons involved in the area are the best people to write a survey. Yes, some people may try to sneak in a survey on a subject of interest only to themselves and three other people, but this should be caught by the committee. (Especially since many of the Oakland PC members will be at the in-person meeting…although maybe I’m just being optimistic about the power of in person meetings.) Gian-Carlo Rota wrote, after all, that “you will be remembered for your expository work.”

    In particular, a survey that takes a “theory” concept with potential important practical applications, clarifies important distinctions between definitions, summarizes best known constructions, and points out “gotchas” could be a key paper for moving that concept from theory to practice. Even if the current best constructions aren’t there yet for practical implementation, if such a survey can highlight the important metrics and open problems, that itself would be valuable guidance for future researchers (and future reviewers). Fuzzy extractors might be a good candidate for something like this, unless a good survey already exists. (Sorry if you wrote it already and I missed it! 😉

    As for page limits, my feeling is that what appears in the conference will in some cases be an “extended abstract,” with details to follow in a full paper. Certainly if one wants detailed constructions and proofs that is not going to fit in a conference paper limit. This raises the concern about how to incentivize people to actually _finish_ the full paper, but I think having an extended abstract of this type is valuable even if the full paper never appears.

    All this just my personal two cents, of course.

    David Molnar

    July 11, 2009 at 7:37 pm

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: