BCOVertigo
@BCOVertigo@lemmy.world
- Comment on Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system— up to 9x increase in output lets 213 GPUs perform like 1,192 5 days ago:
I mean if short term GPU supply is the bottleneck it will help that, but if it cooks the GPUs even faster and doesn’t result in less overall compute being done it just lets smaller players with less hardware compete with hyperscalers and may not actually be savings in the long term. The unit economics being negative are still the giant elephant in the room preparing to trample them.
- Comment on Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system— up to 9x increase in output lets 213 GPUs perform like 1,192 5 days ago:
From their linked study:
“Filling this utilization gap requires us to better saturate each GPU by enabling it to serve requests from multiple models. As such, we aim to conduct effective GPU pooling. By sharing a GPU between as many models as possible with- out violating the service-level objective (SLO), GPU pooling promises great reductions in operational expenses (OPEX) for concurrent LLM serving.”
The “saturate each gpu” part seems to support your idea.
- Comment on RSS co-creator launches new protocol for AI data licensing 1 month ago:
- Comment on Just an fyi for those of you that have the 500gb model and enjoy no mans sky. 1 year ago:
Children render frames incredibly slowly, and usually in crayon. I can’t imagine between that and lack of necessary math skills you wouldn’t get better performance by replacing the family with additional steamdecks of any storage capacity.
- Comment on Just an fyi for those of you that have the 500gb model and enjoy no mans sky. 1 year ago:
You’re doing great lol. Back in my helpdesk days I loved talking to people who overshared to be helpful.
- Comment on Just an fyi for those of you that have the 500gb model and enjoy no mans sky. 1 year ago:
I think that OP is providing all details they think might be relevant, so whether they a little confused or not they got the spirit and I appreciate it.