Category: User Experience


Performance Management at the CRTC

The Canadian Radio-television and Telecommunications Commission (CRTC) website supports the organization’s regulatory activities by providing timely access to public proceedings documents and general interest content particularly suited to consumers of broadcasting and telecommunications services. Its most frequent visitors are representatives of the two industries it regulates, who need the site content for research purposes and to participate in regulatory proceedings.

In the spring of 2009, Systemscope delivered a Performance Measurement Framework and an accompanying research program to support monitoring and reporting on the CRTC site’s performance. The Framework outlined performance indicators with respect to user experience, content and web operations.

Using the Framework as a guide, the Commission worked with a Systemscope analyst to benchmark its existing site performance to understand traffic patterns to the site among new and returning visitors, and then use analytics to improve the user experience.

Case Study – CRTC Performance Management

Yanks Look to Gov Being the Platform for Open Data

by Denise Eisner

Full disclosure: I am a dual citizen of the U.S. and Canada, so any hint of favourtism toward the U.S. approach to open data should be taken with a Memphis-rubbed rib in one hand and a Sam Adams lager in the other.

Open Data is the mantra within government, both here and abroad. In the U.S., government employees, non-profit organizations, developers and community activists are working together on a myriad of data mashups that take rich public data and put it into useful interfaces and onto the platforms people are using. Students with no budgets but seemingly endless spare time are publishing extremely practical applications, while tech start-ups are challenging the notion that data sitting in a dusty archive doesn’t have tremendous value: in fact it can have market value that then translates into new job creation, or what we now call “stimulus.”

The open data fervour reached fever pitch last week at the Gov 2.0 Summit in Washington D.C., where several hundred of the aforementioned groups both from the U.S. and abroad came together to talk about open data, citizen engagement and a slew of related topics. It kicked off with the clarion call by O’Reilly Media’s CEO Tim O’Reilly that “government should think of itself as a platform, to let others build on to deliver additional services to the public.”

With government as “the caretaker for vast stores of information in our national libraries, archives, research laboratories and museums” points out Carl Malamud, founder of Public.Service.Org, those in and outside that conference room should “finish the open gov revolution.” They need open data standards, and need to enforce them. Giving the government’s data to small groups, he says, can turn many small facts into one big truth.

The trouble is that the open data revolution to date has produced a lot of sexy looking government websites, but as Ellen Miller of the Sunlight Foundation notes, not a lot of good data. Data errors and omissions have hobbled many of the U.S. government’s efforts to date. Bloated IT projects with no accountability have undermined efforts to give value to citizens, something that the U.S. government’s CIO is addressing with a deadly serious IT scorecard process that actually halted 30 projects that were in the works and cancelled 12 outright.

What’s the answer? Mr. O’Reilly has it right that government should be the platform, and I would add that the line stops there. Governments should not decide how citizens want their data, but give it freely in formats that can be used across devices and services. ESRI’s president, Jack Dangermond, encouraged government to publish maps as a shared service, one where the data sets are free for anyone to overlay with other data and gain new insights. Developer/blogger Kathy Sierra spoke passionately about creating the killer user, not the killer app. And June Cohen of the powerfully inspiring TED Conferences reached out to the audience to “harness the power and wisdom of crowds.”

David Eaves, the sole Canadian presenter at the Gov 2.0 Summit, stole the show with a fast-paced monologue describing open data successes, all built using available government data. We’ll see David soon at the GTEC conference here in Ottawa as our guest.

As the FCC’s Chairman Julius Genachowski stated, “it’s the public’s data, not ours.”

Denise Eisner is a senior consultant within the Government Service Excellence practice. Follow Denise on Twitter.

Systemscope Volunteer Day: Riding with the Tour for Kids

by Denise Eisner

I’ve done charity bike rides before, including two Boston-NY AIDS Rides in the nineties. That ride lasted three days, covered 434 kilometers and four states, and had the best support crew ever, some in drag. It was the hardest thing I had done in my life, and it represented the kind of challenge that makes you realize that your true potential lies in what you can do, not what you can’t. More importantly, everyone gets a healthy dose of the real big picture.

This year I was again reminded of the big picture on my Systemscope Volunteer Day, when I participated in the Tour for Kids, a cycling odyssey through the rolling hills north of Toronto.

Tour for Kids exists for one reason only — to help children and their families impacted by cancer. Through the generosity of sponsors, the dedication of dozens of volunteers, and the 500 riders who participate, 100% of the donations raised go directly to three camps that let kids with cancer be kids, and heal their hearts and souls toward a full recovery. The ride this year has raised more than $1.2 million, putting a lot of smiles on the faces of children and families that face the daily rigours of cancer.

The challenges faced by a young person with cancer have a strong parallel to the ride. On the ride you face many battles, some small (potholes, flat tires, road kill, rain, heat, cold) and some large (more hills than you can count, aggressive drivers, hail and lightning). There are moments when you want to dismount and throw the bloody bike into the ditch. There are glorious moments when the scenery is breathtaking and your lungs are pumping and everything is working. Then you get a flat. So it goes.

But through it all, you have the support of people around you. On the ride, I had the support of 459 strangers, dozens of volunteers and one wonderful husband. Some were riding 100 kilometers like me, others were going for 160 and the truly possessed rode 200 clicks per day. At the end of each day, we came together for our carb-rich dinner and tear-producing presentations from parents, kids and the professionals who work with these amazing kids at the camps and Sick Kids Hospital in Toronto.

Denise Eisner finishes the Tour for Kids, August 2010

Denise Eisner finishes the Tour for Kids, August 2010

The programs supported by Tour for Kids receive no government money. Government gives the bricks and mortar health care, but the Tour is the emotional top-up that these kids and their families need to keep going. It’s a cause that demonstrates what determination, passion and a common goal can achieve, whether at work or in our personal lives.

We help people like Marisa, shown in this video. God bless her and all the 10,000 kids and their families in Canada braving cancer. You’ll see me back on the Tour next year!

Denise Eisner is a consultant in the Government Service Excellence practice. Follow Denise on twitter

Web Renewal Series: Building a Foundation for Success

by Denise Eisner

Our last post on web renewal examined the different flavours for large scale transformations of a departmental web site. While there are various types and combinations of renewal strategies – content, information architecture and technical – planning their implementation as distinct projects with a beginning, middle and end typically should be regarded as an ominous sign. Treating web renewal as a project means the foundational pieces needed to support the web in a sustainable fashion are weak or perhaps nonexistent. We’ll look at the relationship of web renewal efforts with those pieces to illustrate the optimal path to web renewal success.

Part 2 – Renewal through Constant Change

For large organizations, delivering a web product that is useful and meaningful requires constant change, since the needs and expectations of your users are constantly changing. Large scale overalls of your web presence are sometimes required if you need significant change in direction. The value gained from these exercises, however, will slowly erode if the structures that support the web presence are lacking.

We’ve drawn three simple diagrams to help illustrate why changing your look, content and technology may only get you so far. The first diagram shows an ever expanding gap in your service offering if you take a lights-on approach to your website. The second one shows a similar scenario of service gaps if web renewal projects are carried out in isolation from the management of your foundational structures. The final illustration should be the goal of any organization management a web presence, with the right support in place to evolve alongside the client needs and expectations.

Graph 1

Graph 1

Graph 2

Graph 2

Graph 3

Graph 3

Continual Improvement through the Web Maturity Model

The foundation required to properly manage a web presence can be measured against our Web Maturity Model, which encapsulates a sustainable foundation through seven pillars, each of which plays a critical role in effective Web channel management. The pillars are defined as follows:

Governance – Governance structures define decision-making authority and accountability, typically in the form of persons or groups (committees, boards, working groups, etc.) and are responsible for addressing issues of budget, capacity and ongoing sustainability.

Strategy and Planning – Strategies and plans for the web demonstrate that the organization is making efforts to use the Web effectively, manage and control costs, ensure compliance with relevant statutes and policies and improve service delivery as well as internal business processes.

Roles and Competencies – Developing an effective Web site requires many different competencies and skills working in concert, including:

• Specifying who does what in a given process;
• Specifying the boundaries between different functions in a given process; and
• Specifying accountability for the activities in a given process.

Research-Driven Design – There is increasing recognition that effective Web sites reflect an in-depth understanding of the site’s users and their needs, and that effective Web management requires a commitment to undertaking appropriate research to inform design efforts.

Performance Management – A performance measurement framework defines the means by which the organization will measure success against defined outcomes. The framework should specify which metrics will be used for each outcome, and how the results will be obtained.

Web Standards and Guidance – Almost all federal Web sites are expected to comply with a number of Acts and federal policies, i.e. Common Look and Feel. In addition, standards and guidance should be developed for information architecture, editorial, visual design, IM and technologies.

Technology – The operating systems, applications, programming languages, standards and tools that underpin the organization’s Web development and publishing activities are planned, implemented and evaluated in accordance with desired business outcomes.

Together, these seven components can give your organization the footing it needs to keep pace with the ever-changing needs and expectations of your users and produce a web product that truly delivers. It’s more than just a “web renewal project”: it’s about a program of web management.

Our next post in this four-part series will look at sustainability in web channel management as an antidote to the resource-intensive web renewal project.

This blog was written by Denise Eisner with support from Alexandra Katseva and Kellen Greenberg.

Web Renewal Series: 6 Departments, 3 Consultants, 4 Blogs

by Kellen Greenberg

Recently my colleagues Denise, Alex and I met up at the local diner to catch up on our projects. Over eggs and coffee we realized that despite us working on Web-related projects in six different departments, the issues, challenges and at times high stress levels faced by our public sector clients are very similar. We decided to put together a series of blogs to pull together some of the important lessons learned from this aggregate of projects.

This series of Web Renewal blogs is designed to help the Federal Government continue in its quest to deliver meaningful web content to Canadians.

Part 1 – More Than One Piece to the Puzzle

All six of the Web Renewal projects we were working on tackled seemingly similar challenges under similar labels: Web Renewal, Web Renovation, Web Transformation. However, when we lifted the rock to see what these projects were really about, we were presented with a host of different challenges. The rubric of “Web renewal” (or something similar) is being used as an umbrella for what we’re seeing as four different types of project:

  • IA Renewal: Often the stepping stone to a deeper problem, clients are looking to fix a broken Web experience by cleaning up their navigation and underlying information architectures.
  • Content Integration: A lot of our Government clients are still working hard to clean up their footprint on the Web. We’ve helped several organizations forge ahead with taking their 200+ websites and streamline them into a single, cohesive Web presence.
  • Content Renewal: This type of work involves taking all of walls of policy text from your website and turning into useful, findable, readable, meaningful Web content. If only people would spend more time and money in this area!
  • Technical Migration: Moving from one technical platform to another. The panacea of managing and delivering content through a Web Content Management System (WCMS) (or a shinier newer one), often gets embedded with cleaning up content and navigation.
  • None of the above: In some cases, the issue being manifested on the Web is rooted in issues that have very little to do with the Web. Absence of organizational vision or direction, poor leadership, misunderstanding of client needs, or complicated policies and operations can all be brought to the surface by trying to make a usable Web experience. However, just because the website is unclear, doesn’t the mean the problem is with the Web.

What’s the point of all these definitions? Fixing a large GC website can have many different facets to it, and moving forward towards a fix requires a good understanding of what the true problems are, a solid project plan and the support of senior management.

Mixing up the right Web Renewal cocktail from the outset can save a lot of pain and frustration down the road. If you’re looking onto a similar type of project for your organization, here are a few quick notes about how to best shape your project and avoid shooting yourself in the foot.

  • Content Integration and Information Architecture work are a natural fit together.
  • Content Integration and Content Renewal are also a good pairing.
  • Content Integration, Renewal, and IA can all be done at once too, but it’s a big project.
  • Content Renewal on its own or combine with other things is a lot of work. It’s incredibly valuable and necessary work, but a lot of it.
  • Technical Migration is just different. Doing this in lock-step with the uprooting your IA and content can be extremely challenging.

Our next post in this series will focus on some of the foundational pieces often overlooked in Web Renewal projects that can make or break its success.

This blog was written by Kellen Greenberg with support from Alexandra Katseva and Denise Eisner.

Website Rx: content audit

By Denise Eisner

Do you find yourself looking at your organization’s website and wondering: why can’t we just throw it all out and start over? Wouldn’t that be better than working with stale content that no one reads? Or that was architected by previous management who have moved on?

We’ve yet to see a government department act on this impulse, which is fortunate. Starting from scratch undoes months of work and means throwing the baby out with the bathwater. There’s likely some nugget of good content lurking here and there, and more importantly, some relevant thinking about audience segments, content types and tasks. So what is the logical next step for an ailing site?

A web strategy is always a good start. It has the benefit of setting a direction that then guides all future development. Another more hands-on effort (which can occur simultaneously) is a content audit. An audit involves conducting an inventory, identifying ownership and then having owners make decisions about their contents’ future.

Environment Canada recently used a content audit approach to sift through 100,000 pages. They ended up with a much leaner content repository that now stands at a third of the original. By getting rid of the stale content, the relevant content is easier for users to find

During an audit, you want owners to make decisions about their content. No owner willing to step up? Archive the content and take it off the site. No owners willing to make decisions? Give them plenty of notice (and gentle reminders) and explain the end goal (better site management, satisfied users, etc.) but give them a deadline you’re ready to act upon. As in, you’ll remove the content unless they decide. And do so. A reaction sets the stage for a fulsome discussion of the content value relative to the goals of users and desired business outcomes.

If you’re looking at a site with more than 500 pages, this is a full-blown project, requiring a dedicated coordinator and one or more other team members to talk to content owners and catalogue the results. The effort in days will depend on the number of pages, owners and complexity of the site structure. You’ll need well-written criteria to make content decisions and a communications plan to articulate the project to various stakeholders. Management should regularly be apprised of your progress and alerted of any issues or risks requiring their action.

A content audit sounds like tough medicine but it’s the most effective way to deal with sites that have lost their way, without throwing the good pieces in the trash by accident.

Denise Eisner is a consultant in the Government Service Excellence practice. Follow Denise on twitter

Of Clocks and Clouds: The Right Support For Your Processes

In the May 2010 edition of Wired, Jonah Leher’s article got me thinking about philosopher Karl Popper and how his theory on Clocks and Clouds isn’t far off an approach I use when designing processes. Popper:

“..divided the world into two categories: clocks and clouds. Clocks are neat, orderly systems that can be solved through reduction; clouds are an epistemic mess, highly irregular, disorderly, and more or less unpredictable.”

Processes are of clocks and clouds too.

read more

If I Could Be a Government Web Site Manager

By Denise Eisner

I work alongside some very dedicated, passionate web people in government. They want to have web sites that are usable, readable and of value to the audiences they’re intended to reach. For reasons both obvious to those in government and to anyone else who has tried to push new ideas in large organizations, there’s a set of what we euphemistically term “challenges” to achieving these goals.

But let’s say for a moment that said challenges were surmountable and the HIPPO (Highest Paid Person in the Office) was giving me carte blanche to run the web site per accepted best practices and the latest in research-driven design principles. Wow! Colour me happy!

Quick, before the HIPPO changes his/her mind, here’s my wish list:

  • Double the size of the web team – No site that is designed to reach audience segments as vast and varied as those served by government sites can be run with two people, neither of which have time to strategize, plan, write, edit, apply metatags, code, test, and perform quality control, all while responding to the latest request to convert a 200-page report to HTML. Scale the team to the size of the site really needed by users.
  • Let my team control the site design – Everyone has opinions but design by opinion war only leads to chaos and bad feeling. I’ll consult with stakeholders, sure. I’ll amass quality research to back up my ideas and proposals (web metrics, surveys, usability tests, etc.). But I’ll make all high-level design decisions regarding navigation, breadcrumbs and landing pages. And I’ll be able to defend those decisions with data.
  • Have dedicated IT resource(s) on the Web team – Rather than have the disconnect that can exist between varying business units, combine the various skill sets needed to have a strong web group capable of supporting high-quality content and web infrastructure.
  • Publish good content, not FAQs – I was recently inspired by R. Stephen Gracey’s post on how FAQS “seem to constitute a basic instruction manual or else call attention to selling features, making them only marginally useful to users with real questions.” I want good writers to develop quality, searchable content and an editor to oversee publishing standards.
  • Help me get the web strategy approved – I need senior support for defining why we need a web site and who we really serve (beyond the catch-all “all Canadians”). This will help me maintain a focused web operation that strategic, not reactionary, and supports our business priorities.
  • Approve a governance model for the web – In order to make informed, strategic decisions around the web, particularly for the aforementioned strategy, let’s implement the roles we defined for a web champion, working group, ad hoc teams and steering committee.
  • I’ll just do Web 2.0, now – Hey, I’ll start a blog! I found a SME who’s willing to share his/her expertise with a specific audience (teachers, businesspeople, scientists), so I added the blog to the site, moderated it myself, and can report the site activity to management. It involves extra work but as EPA web 2.0 guru Jeffrey Levy told me last year, you learn by doing. We’ll keep an eye on performance and keep tweaking it as needed.
  • Good measurement tools – You only can manage what you can measure. Let’s get the right tool and get a professional to configure it according to our performance indicators. I can help find efficiencies if I have good data to present to senior management.

Quite a wish list, but these approaches all point to effective site management.