Write about the evaluation

From Organic Data Science Framework
Revision as of 16:44, 29 September 2014 by Felix (Talk | contribs)

Jump to: navigation, search


Possible dimensions for evaluation:

  1. show how easy it is for new people to participate. For example, show total training time, show how often they go back to consult documentation, show how often they delete things they have created because they made mistakes, etc. Could also do a survey of new users. New users will include: Jordan, Craig, Hilary, Gopal (anyone else?).
  1. show how people find relevant tasks. For example, show how they use search, how they get to tasks they want to do, etc.
If the tracked data has a certain variation we could represent the conversation rate of components.
FindingRelevantTasks.png
We can add additional heat maps to illustrate where the users focus on.
Heatmap G16 Workshop.png
Heatmap Mainpage.png
Heatmap FrameworkDesign.png
  1. show how people track their own tasks. Here we would do a survey to ask how easy it is for people to track tasks.
Survey with categories 1 to 10:
  1. My Task Tab
  2. Task Alert
  3. Person Page
  4. Expertise

Need to show collaboration touch points:

  1. is more than one person signed up to each task?
Nr of Participants #Task Tasks in %
0 11 9.01%
1 27 22.13%
2 33 27.04%
3 36 29.50%
4 14 11.47%
>4 1 0.81%
Data Source: JSON Serialized Tasks.
PersonPercentageGraph.png
  1. is more than one person editing the metadata of tasks?
Nr of Meta Data Editors #Tasks Tasks in %
1 136 75.98%
2 32 17.88%
3 6 3.35%
4 3 1.68%
>4 2 1.12%
Data Source: Tracking Data.
  1. is more than one person editing the content text of tasks?
Nr of Content Editors #Tasks Tasks in %
1 210 87.50%
2 25 10.42%
3 3 1.25%
4 0 0%
>4 2 0.84%
Data Source: Wiki Logs.
  1. is person collaborating with more than one other person?
Data Source: JSON Serialized Tasks.


Collaboration network: Every node represents an organic data science user. The strength of collaboration between users is expressed with strength of each edge.
PersonNetworkv0.1.png


Yandex.Metrica