SAve MY nEOCiTy! - Neocities site downloader <3

[Melooon]

Hello! I'd like to share my neocities site archival tool! I noticed a lot of people have trouble downloading their sites so I put this lil app together!

It's really handy to be able to download your site in order to save multiple copies of it (particularly if you're planning a redesign); in order to make local edits if you're gonna be working offline, or if you'd like to move to a new web host!

It's available for Mac, Windows and Linux!

Features:

• Downloads ALL site files, even unlinked ones!
• Works even if the official Download button is missing from your profile!
• Optional folder time-stamping for site archives!
• Attempts to re-try unsuccessful downloads!
• Provides real-time feedback!
• Provides a results log including any files it fails to archive!

Screenshot of v0.0.1


Downloads & Links!

You can download on my site here 
https://melonking.net/melon?z=free/software/neocities-downloader

You can also look at the source code here 
https://github.com/Melonking906/SAve-MY-nEOCiTy

[keybladekid6]

!!! thank you so much for making this!! i was actually planning on redownloading all of my site files manually recently and it would have taken forever, i'll try this out right away 

is there any particular pattern to the uppercase/lowercase letters? I tried spelling out a word with them but couldn't figure it out 

xoxo, lolo =3

[Capybara]

Thank you so much for this!!! I've been stressed out over wanting an offline copy of my site but most of my backups are outdated, my site is too big to download on Neocities itself, etc. but I'll use this tomorrow!

[tarraxahum]

Just tried it out, works like a charm! Thank you so much Melon, now backuping will be SO much easier <3

Loved the soundtrack, by the way. Kinda made me feel in some old-school hacker movie 

[wygolvillage]

This tool is perfect! I tried it out today since it was about time I make a backup. It's so easy to use :O

[BlazingCobaltX]

Lifesaver! I literally went to bed last night thinking I have to backup my site, and this tool did it fast and properly.

[j]

this is really cool! i love the UI design :)
i've got to say: you always make programs way more artsy than i ever could - i'm a little jealous @Melooon!

you inspired me to make something similar too - as a little challenge for myself. i'm more of a minimalist terminal geek myself, so here's something you can run in a terminal if you prefer just bash!

Code

#!/bin/sh

read -p "USERNAME: " USERNAME
read -p "PASSWORD: " PASSWORD
read -p "SITE-NAME: " SITE_NAME
DATE=$(date '+%Y-%m-%d')

CREDS_OK=$(curl --output /dev/null --user "$USERNAME:$PASSWORD" \
--url "https://neocities.org/api/list" --silent -w "%{http_code}")

if [ ! "$CREDS_OK" == "200" ]
then
    echo "Problem: $CREDS_OK"
    exit
fi

SITE_OK=$(curl --output /dev/null --url "https://$SITE_NAME.neocities.org" \
--silent -w "%{http_code}")

if [ ! "$SITE_OK" == "200" ]
then
    echo "Problem: $SITE_OK"
    exit
fi

mkdir -p "./$SITE_NAME-$DATE"
curl --silent https://neocities.org/api/list \
--user "$USERNAME:$PASSWORD" \
| grep "path" | cut -d '"' -f4 | \
xargs -I {} curl --silent -o "$SITE_NAME-$DATE/{}" \
--location --url "https://$SITE_NAME.neocities.org/{}"

echo "Success"

[Melooon]

so here's something you can run in a terminal if you prefer just bash!

Wow getting it down to 33 lines is incredible! Although the USERNAME and SITE_NAME are always the same (even when you have multiple sites), so you could go smaller

Id love to see more handy tools like this; the other cool lil bash script that could be done is one that returns the total site size based of the list! (people seem to enjoy knowing that info)

is there any particular pattern to the uppercase/lowercase letters?

I just did it like that to be kool I didn't consider anyone would go looking for deeper meaning 

[j]

Wow getting it down to 33 lines is incredible! Although the USERNAME and SITE_NAME are always the same (even when you have multiple sites), so you could go smaller

thanks! definitely don't look under the hood at the line counts of the tools i used (shell script, read (libc), curl etc...) :P

you're right about the username and site_name stuff, but i tend to use my email (instead of my username) to login to neocities - which would make fetching URLs a little difficult if i were to use the same variable:

Code

https://email@provider.neocities.org

... so unfortunately there's a need for both!

Id love to see more handy tools like this; the other cool lil bash script that could be done is one that returns the total site size based of the list! (people seem to enjoy knowing that info)

haha! don't tempt me! :P

that being said - the neocities API doesn't offer anything like this; you'd still have to download the entire site, then run `du` to get the size of the directory - i suppose it's a two birds, one stone scenario?

[Melooon]

that being said - the neocities API doesn't offer anything like this

Actually, it does! When you do /api/list you get a list of all files, but each file also lists its size in bytes; you just have to iterate the list and add up all the sizes (I do this in my app to give download feedback) - so you can get the entire site size with a single API call, no download needed!

[j]

Actually, it does! When you do /api/list you get a list of all files, but each file also lists its size in bytes; you just have to iterate the list and add up all the sizes (I do this in my app to give download feedback) - so you can get the entire site size with a single API call, no download needed!

i didn't realize! i was looking for an individual endpoint specific to that purpose - but this makes more sense. might add it in which case; thanks!

[Corrupted Unicorn]

YEAH!! Gotta try this soon. I make a backup of my website manually but this WILL come in handy.

[georgemoody]

Thanks to this little tool I moved my website from neocities to envs.net! thank you Mel! 

[Corrupted Unicorn]

Hey, may I ask, what is the password requested for?

Not that I don't trust you dear Meloon, but my old-school Internet upbringing is rearing its head and bleating "NEVER EVER SHARE YOUR PASSWORD WITH ANYBODY!" so it would be good to know why is it being asked for and what it is done with it.

[sig]

Hey, may I ask, what is the password requested for?

I assume it's to verify you're the owner of the site so that you can't just rip any old Neocities page?