Project Evaluation

class: center, middle

# CSCI-UA 480.10: OSSD

<br>

## Project Evaluation
### (or, is a project right for you?)

.author[
Instructor: Joanna Klukowska<br>
]

.license[
Adapted from _[Project Evaluation](http://www.compsci.hunter.cuny.edu/~sweiss/course_materials/csci395.86/slides/project_evaluation.html#1)_ by Stewart Weiss <br>
Unless noted otherwise all content is released under a
 [Creative Commons Attribution-ShareAlike 4.0 International License](https://creativecommons.org/licenses/by/4.0/).
]

---

## What to Consider: A Synopsis

- Project documents

--
   - project website: what information can be found on the project website? can you get to the repository from the website?
   - in the repository: what documents are there? how complete are they? how hard to follow?

- Project tools and technology

--
    - what issue tracker do they use, what communication channels,
  what languages, tools in general, development environment

- Project activity

--
    - how active is the community, lots of issues and pull
    requests, many questions asked and answered
--

- Issue tracking

--
    - are issues triaged and tagged, are they marked for newcomers, do they all
     look hard
--

- Community responsiveness

--
    - are pull requests merged quickly, are issues talked
  about, are questions answered timely

- Community welcomeness

--
   - is the community welcoming, is it newcomer-friendly,
   is the conduct helpful and respectful

---

## The Project  Documents: Project Website

- Start on the project website

- some projects keep a lot of documents/instructions there

- how easy/hard is it to find info about contributing?

- what other information is there that may be useful to the developer?

---

## The Project  Documents: Project Repo

- Make sure they are all there. That is not enough, though. Need to inspect them.

- `README`: __Read it!__ Is it
   - easy to follow?
   - designed for beginners to understand?
   - thorough - says where to find everything else, etc?

- `LICENSE` (and/or `COPYING`)
   - Without it the project is not open source. __Pick a different project.__
   - Which license? Are contributors expected to sign CLA before making a contribution?

- `CONTRIBUTING`: __Read it!__ Is it
   - easy to follow?
   - designed for beginners to understand?
   - thorough - has style guides, coding guides, all procedures detailed?

- `CODE OF CONDUCT`: __Browse it!__
   - If it missing it is not a good sign. __Pick a different project.__

---

## Other Project Documents?

- Are there other important documents that can make or break it for you?

Maybe. What about:

- `INSTALLATION`
    - Are the instructions easy for newcomers?
    - Do you understand them?
    - Installation instructions for users and developers are often different:
	is there a distinction?
    - If it is a real candidate, try to install it. If you can't do so easily,
	__pick a different project.__

---
## Tools and Technology: Code

- Various questions about the code and languages:

--
  - Can you program in the primary languages of the project?
  - Can you read code in those languages?
  - How hard is it to understand the project code? Is it a large code base?
		- look at some actual code files
		- is it easy to find the code?
		- is the code documented to help you understand what it is doing?
  - Does the project depend on external additional software modules such as
    graphics libraries?
		- if so, are there easy to follow instructions for installing those?
		- try to install those libraries
  - Do you need to have specialized knowledge to work on this project?
  		- for example, understand AI algorithms or understand astro-physics

---
## Tools and Technology: Issue tracker

- Find the issue tracker.

--
  - Is it in the repository or elsewhere? How easy is it to find it?
  - Are there open issues? How many?
  - Is it triaged?
  - Are issues tagged?

---
## Tools and Technology: Communication

- What are the asynchronous communication channels used in the project?

- Do they use email and/or mailing lists?
   - Do they use IRC or some other type of chat that has an asynchronous mode
   such as _Slack_, _Discord_, etc?
  - Is there a discussion board?
  - Are there instructions for how to join the communication channels?

---

## How Active is This Project?

- This is the million-dollar question. You need a way to assess how active it is.

- Many questions to answer.

--
  - How recent are the  last few commits?

--
  - How many people are maintaining the project?

--
  - How many contributors has the project had in the past year?
    in the past few months?

--
  - How long do issues stay open? (too long may be a bad sign, too short is also not good)

--
  - How long do pull requests/merge requests stay open?

--
  - How many issues are currently open?

--
  - Is there a lot of discussion about issues and pull requests?

- Stay away from projects with small staff with long periods of inactivity.
  These people have other obligations that keep them from handling
  your contributions in the time frame you want. It might still be a great
  project to work on, but not for the first contribution and not for a class
  that has a fixed time frame.

---

## Issues and Issue Tracking

- You want a project that makes it easy for you to jump  in and find a good
  issue on which to work.

- Are the issues tagged for easy filtering?

- Are they marked with tags like _First-Timers Only_,
   _Newbie_, _Good First Issue_, _Easy_, and so on?
   	- if so, look at some of those issue, do they seem like something a newbie could do?

- Are there detailed descriptions, and do they require very detailed
   descriptions?

- Do they have an __Assignee__ field so that you will know if they are taken? <br>
  OR Are the issues being assigned to particular contributors in some other way?

- Are there long discussions that can be read for some of the issues? Read them!
    Read the conversations about __closed issues__. This gives you several insights:
		- how should you use the issue tracker? (is it only for reporting bugs? can it be used for asking questions? what format should the issues be in?)
		- who are the people that answer the questions and/or comment on issues?
		- what is the tone of the conversation? are people kind to newbies? are people helpful?

---
## Community Responsiveness

- Responsiveness is not the same as welcomeness. How long does it take to get a
response, regardless of its tone?

- Do the people in this project  respond quickly to questions in general,
   such as how to install, or a question about ambiguous documentation?
   		- this could be in the communication channels, or in the issue tracker

- Do the people respond to questions about existing issues?
	   - this should be in the issue tracker (or in the communication channels)

- Do they respond to submitted pull requests or to questions before pull
   requests are submitted?

- In conversations, do people address the questions or do they skirt them?

---
## Community Welcomeness

- You should not become involved in a toxic community or in one that _feels_
  toxic to you. __Read the various discussions and conversations, especially the
  long ones.__

- Look at the closed issues and read their conversations.
--

- Is the project's community welcoming? Are their communications not just
   polite, but friendly, and maybe even __understanding__?
--

- Are responses generally constructive?
--

- Do the conversations and discussions adhere to the Code of Conduct?
--

- Do people use civil language?
--

- Do they refrain from  personal attacks?
--

- Do maintainers thank people for their contributions?

- Are you afraid to reach out, or are you intimidated? You need to decide if it
is because of something in the community or if it is just your own anxiety or fear
or shyness.  (If it is the latter, you must overcome it and see what happens.)
	- Remember, _[On the Internet, nobody knows you're a dog](https://en.wikipedia.org/wiki/On_the_Internet%2C_nobody_knows_you're_a_dog)_

---

## Is That It?

- After you consider all of the facets of a project and its community, you still
  need to decide if it is a good fit for you.

- This is where your goals, skills, resources, and interests enter the picture.
  Is this project something __*in which you are really interested*__?

- If not, find a different project because your lack of enthusiasm will win.

- Do you have the skills to tackle some of the issues that you found? Are
  there enough open issues that you'll be able to find one or two?

- If you do not have the time to achieve your goals in this project because
  there is just too much to learn, find a different project or modify your goals.

---

## What to Consider: A Synopsis

- Project documents
   - project website: what information can be found on the project website? can you get to the repository from the website?
   - in the repository: what documents are there? how complete are they? how hard to follow?
- Project tools and technology
    - what issue tracker do they use, what communication channels,
  what languages, tools in general, development environment
- Project activity
    - how active is the community, lots of issues and pull
    requests, many questions asked and answered
- Issue tracking
    - are issues triaged and tagged, are they marked for newcomers, do they all
     look hard
- Community responsiveness
    - are pull requests merged quickly, are issues talked
  about, are questions answered timely
- Community welcomeness
   - is the community welcoming, is it newcomer-friendly,
   is the conduct helpful and respectful

---

## Let's look at some projects

- [Coala](https://coala.io/)

- [Exercism](https://exercism.org/)

- [AnitaB.org](https://anitab.org/),

--
    [AnitaB.org on GitHub](https://github.com/anitab-org)

---
## Sources

1. VM Brasseur, _Forge Your Future with Open Source_, The Pragmatic Programmers, LLC. 2018.
2. https://opensource.guide/how-to-contribute/#finding-a-project-to-contribute-to