I just saw, that fedidb now has data for the biggest fediverse accounts, so I did a little plotting with it. Here is a graphic of the scattering of the 100 biggest accounts by the instance they are on. 38 of them are on mastodon.social https://fedidb.org/popular-fediverse-accounts (the data is in the alt text) #mastodon #chart #fedidb #fediverse
Made a little script and raked almost 1800 servers and over 40.000 communities.
If someone has the http api call for messages and posts I’d love continuing and maybe set up some sort of search engine… Or maybe I should go with the soap(?) one but I don’t know how to do it in python, any information greatly appreciated!
oh, Good luck with that. Make sure however to respect the users privacy and indexing preferences. People in the Fediverse are very privacy consious and not everyone likes their post scraped and indexed.
I’d start with the Mastodon docs, it’s a solid resource to get started.
Hmm… I’d only index public data (I’m not totally there yet of course), which can be found by anyone, but if there is some way for people, posts, communities, servers, to opt out then ok. Serious question: is there though? Second question, I’m wanting to do this open source, which means anybody can take it, remove the check and scan everything. What are your thoughts about that?
If you know how to query servers, communities, posts or comments on that topic I’m all ears, I’m only doing 50% of that today BTW.
On a side note, where is your 0xCAFE come from? Is it like the stack overflow/ memory error checks like 0xDEAD(or 0xDEADBEEF) and so?
Cheers!