The sample efficiency of the RL algorithm, even for simple games, is not very good. This usually means that we will need a lot of episodes for the policy to learn to excel. Being able to run policy in an environment that can parallel and accelerate could be very helpful for the improvement - for example running a batch of browsers or tabs simultaneously :)
People born in different eras often develop different worldviews by the time they reach their 50s. Not everyone is lucky enough to be born in the "right time".
I am curious about the setup of 14 GPUs - what kind of platform (motherboard) do you use to support so many PCIe lanes? And do you even have a chassis? Is it rack-mounted? Thanks!
I used a large supermicro server chassis, a dual Xeon motherboard with 7 8 lane PCI Express slots, all the ram it would take (bought second hand), splitters, four massive powersupplies. I extended the server chassis with aluminum angle riveted onto the base. It could be rack mounted but I'd hate to be the person lifting it in. The 3090s were a mix, 10 of the same type (small, and with blower style fans on them) and 4 much larger ones that were kind of hard to accommodate (much wider and longer). I've linked to the splitter board manufacturer in another comment in this thread. That's the 'hard to get' component but once you have those and good cables to go with them the remaining setup problems are mostly power and heat management.
1. Start by learning a simulation tool, e.g. Mujoco (open source) or Isaac Sim.
2. Learn basics of optimal control and reinforcement learning, reproduce papers/ideas in the simulation.
3. Get your hands dirty on a cheap robot, and try deploy your trained model on it. For mobility and manipulation. Unitree Go1/Go2 for mobility, and robotic arms for manipulation.
I think having to debug to find problem of your system is frustrating. But with NixOS, I at least won't be afraid of "breaking the system" or doing something "irreversible". This is totally a peace of mind when tinkering with my setup.
Wow, what a fantastic write-up—thanks for sharing this! I’m a San Jose homeowner (and PG&E sufferer) with a homelab that pulls over 1 kW, and I’ve been down the DIY solar rabbit hole for the past two weeks. Based on my research, I’m planning a roughly 9 kW Signature Solar setup:
18 U server rack (~$500)
— total hardware ~$14 760
My big hang-up has been the rooftop work, permitting and inspections—almost no one I call will touch a true DIY system. If anyone here in the Bay Area has recommendations for installers or back-of-house permit-whisperers who’ll partner on a non-Tesla/Sunrun job, I’d love to hear how you made it happen. Thanks again for the inspiring guide!
Greenlancer will draw up code-compliant plans that you can submit to your local building permit agency, and they'll revise if anything needs it. It cost less than $400 last year. You've done enough research that they'll be able to easily take your project and turn it into something legal.
I recently did an Enphase system of a similar size to yours. It was fully DIY except for wiring the combiner and a roofing company to plug all the holes I drilled. Working with PG&E was truly an epic year-plus battle culminating in a CPUC complaint, but in the end it was really just a bunch of emails.
I don't have any installer recommendations, but it should be easy enough to find a local electrician, and I've found that they tend to know others in adjacent fields.
Thanks so much for sharing your story – hearing about your DIY Enphase install (and epic PG&E battle!) really gives me confidence. And the information you shared is extremely helpful for first-time DIYers like me.
reply