Matthew Gifford

Sessions from this year’s Gnomedex will be streamed.

• Sticker on presenter’s laptop: My other computer is a data center.
• Top ten ways to sabotage your project with svn
  • Argue about version control systems
  • Do a brute-force transition
  • Backups? What backups?
  • Loads of locales
  • Rule with an IRON FIST
  • Hide the version control
  • Use complex branching schemes
  • Put everything in the repository
  • Use a network drive
  • Really clever hook scripts!
  • Edit the repository database

• Imagine a world in which every single person on the planet is given free access to the sum of all human knowledge.
• Wikipedia: encyclopedia
• Wikia: the rest of the library
• 3000 communities
• Wikia is a completely separate organization from Wikipedia
• Open source has made the barrier for launching a site very low
• Search is next
• search.wikia.com
• Transparency - Openness in how the systems and algorithms operate, both in the form of open source licenses and open content and APIs
• Collaboration - Everyone is able to contribute in some way, strong social and community focus
• Quality - Significantly improve the relevancy and accuracy of search results and the searching experience
• Privacy - Pursuing the Holy Grail of privacy protection
• Grub
  • is being open-sourced
  • distributed crawling
  • Windows-only now
• Restaurant analogy
  • To serve steak, you need knives
  • Some people with knives will stab other people
  • To solve this, some would put each person in a cage
  • This makes a bad society

These notes are incomplete because most of the talk involved code examples. Amy is posting the slides on her site.

• JavaScript is a real langugage.
• Browser support and dev tools are much better than they used to be
• Everything’s an object
• Prototype allows you to extend objects that are already instantiated
• Prototype objects do not override attributes that are already designed
• prototype.js extends the Object object
• prototype.js has a Class object

• Why offline?
  • Lack of always-on internet access
  • Travel
  • ISP service interruptions
  • Remove; unconnected land and water
  • More and more data lives in the cloud
  • Your users are asking for it
• Why offline AJAX?
  • Data closer to the user
  • Performance
  • Clear, simple programming model
  • Same UI both online/offline
• What solutions are out there today?
  • Fancy caching approach
    • Firefox 3 offline support
    • Google Gears
    • Dojo Offline Toolkit
  • Desktop VM approach
    • Mozilla XUL Runner
    • Adobe AIR
    • Joyent Slingshot
  • Roll your own
    • Salesforce.com
    • TiddlyWiki
    • Zimbra Desktop
• Firefox 3 offline
  • Features
    • Anywhere Firefox runs
    • Gecko
    • WHATWG client-side storage, offline events
    • SQLite for offline storage
  • License
    • MPL
  • Availability
    • Alpha 5
• Google Gears
  • Features
    • LocalServer
    • Database
    • WorkerPool
    • Firefox, IE
  • License
    • New BSD license
  • Availability
    • Public beta
    • Google Reader Offline
    • Several sample apps
• Dojo Offline Toolkit
  • Features
    • Built on Google Gears
    • Extends Gears with higher level API
    • Encryption support
    • Simple hashtable storage API
  • License
    • AFL or BSD
  • Availability
    • Beta with Dojo 0.9
    • Live demos
• Mozilla XUL Runner
  • Features
    • Where Firefox runs
    • Gecko rendering
    • Javascript access XPCOM objects via XPConnect
  • License
    • MPL
  • Availability
    • Developer preview
    • Shipping apps today (Google AdWords Editor)
• AIR
  • Features
    • FileIO, on/offline, windowing, clipboard, drag and drop, local storage, network APIs
    • Eventually could be delivered via Flash
    • WebKit rendering
  • License
    • Plan to contribute rendering changes to WebKit
    • Runtime free but closed source
    • Developer tools cost
  • Availability
    • Pre-release beta
    • Several demo apps
• Joyent Slingshot
  • Features
    • Ruby on Rails
    • ActiveRecord Sync
    • SQLite local storage
    • Seems to lack OS native support
  • License
    • GPL v2, commercial version
    • Free if using Joyent’s application hosting
  • Availability
    • OSS public release
    • Several apps ported
• Zimbra demo
• Zimbra Offline
  • Features
    • Linux, Win, Mac
    • Firefox, IE, Safari
    • Identical UI online/offline
    • Micro server for sync, persistence, and search
      • Derby (meta data, user profile)
      • Lucene (full text search)
      • Jetty (HTTP, JSP)
      • Filesystem for messages
    • Support for large datasets
  • License
    • MPL/ZPL
  • Availability
    • Zimbra Desktop Alpha
• Challenges for the developer
  • Select what to take offline
    • Security risks
    • Does the user need it?
    • Can they use it offline?
  • Sync is hard
    • Conflict resolution
    • Multi-user and multi-client
  • End-user desktop support
    • Risk of increased support/debugging costs
  • Upgrades and patches
    • Have a plan before you release

• Do databases matter?
• What are the characteristics of the ideal database?
  • Invisible
  • Available
  • Reliable
  • Scalable
• Oracle 11g new features
  • Enhancements to Data Guard
  • Flashback and Total Recall
  • Real application testing
  • Partition enhancements
  • Oracle OLAP enhancements
  • No core performance improvements
• Choosing the right open source database: discussion of differences
  • Application’s data access characteristics
    • Primarily read access
    • Mix of reads, inserts, updates, deletes
  • App dev technologies
  • App deployment characteristics
    • Platform
    • Performances
    • Scalability
    • Availability
    • Security
    • Maintainability
    • Footprint
• Common Misconceptions
  • ANSI SQL Standard means database apps are portable
  • Performance benchmarks are relevant
  • Open source databases are immature
  • Open source databases are not secure
• The best line of code is the one you don’t have to write.
• Application design considerations
  • Performance
  • Availability
  • Scalability
  • Reliability & Robustness
  • Deployment
  • Maintenance
  • Usability
• Ingres
  • Overview
    • roots in open source at UC Berkeley
    • Ingres goes commercial and mistakes were made
    • Returning to our open source roots, why and how?
    • Who uses Ingres?
    • What does Ingres offer?
      • Performance
        • Parallel query support
        • Table partitioning
        • Advanced query optimization techniques
      • Availability
        • Database clusters
        • online table and index reorganization
        • Online backup
        • Peer-to-peer replication

• What is Twitter?
  • Social networking/RSS/SMS
  • Ruby on Rails app
  • Horozontially scaled
  • Centralized MySQL backend
• Twitter’s problem
  • Started with a few users
  • Didn’t scale with their success
  • High latencies could make it unusable
  • Possible suspects
    • OS: kernel libraries, etc.
    • Ruby, MySQL, Apache
    • The app itself
• Enter DTrace
  • Twitter had DTrace
  • Systemic observability
  • Concise answers to arbitrary questions
  • Designed for production systems
  • Also designed for developers
• Demo
• The Investigation
  • High latencies under load
  • What could cause these latencies?
    • on-CPU time
    • off-CPU time
• Stepped through the Twitter investigation
• DTrace your application
  • Get your app on an OS with DTrace
  • Start with the tools you know, dive deepter with DTrace
  • DTrace sees all

I just couldn’t help but post this. What an embarrassment.

These notes are incomplete. Just wrote a couple things that struck me.

• Epiphany in innovation is a myth
• All innovation displaces something else. You have to be able to overcome the reaction to the displacement.
• Good quote from William McKnight.

I was too busy paying trying to follow the code to write notes. Here are the slides.