||2 years ago|
|.github||2 years ago|
|etc||2 years ago|
|examples||2 years ago|
|private||2 years ago|
|.gitignore||2 years ago|
|LICENSE||2 years ago|
|README.org||2 years ago|
|info.rkt||2 years ago|
|main.rkt||2 years ago|
What is Feed Analyser?
This aggregates content from different channels(for now only twitter and github commits are properly supported) and displays them in one place. In most hackathons, organisations and sometimes events, content spans different places e.g. Discord, slack, IRC, mailing list, issue trackers etc. This project aims to aggregate news from different places and display them in one place.
For now, the project supports twitter and github.
It also allows you to vote on the feeds. You can view a demo here
How does it work?
The program scrapes content from twitter and github and stores the data into redis queues with a score. You can then read from the redis queue.
There's also an in-built provision for voting on queues, whereby each vote updates the zscore of an individual item thereby increasing or decreasing the likelihood of it being displayed on a page.
Why are you storing things to Redis? Why not just generate pages straight from them?
We'd like to extend the program later so that it can provide a source of aggregate data to other programs. To achieve this, you'd need an intermediary data store, and for this, Redis was chosen.
How do I run this?
First ensure you have twint installed- this is used to fetch data from twitter without using twitter's restrictive API.
Start the twitter/ gh fetching daemon:
# Replace example.conf.rkt with your own conf file with the correct settings racket examples/update-feed.rkt -c etc/example.conf.rkt
To run the server
# Replace example.conf.rkt with your own conf file with the correct settings racket examples/polling-server.rkt -c etc/example.conf.rkt
Suggestions/ Help Wanted
Fetch data from Slack
Come up with an algorithm to sieve out noise(Machine Learning)
Expose content using an API
Come up with a sane deployment process
[x] Better Parsing from Twitter
Add package and dependencies to Guix
Integrate to GN2
Fetch Pull Requests and merges from GitHub and display them
Fetch Data from Slack
Make a lib layer and add project to upstream to racket-lang.org
Fetch Data from IRC
Add NLP to make more meaningful data