The recently introduced Primer model reduces the computation needed to achieve the same accuracy by 4x. Other studies examined training the Transformer model on an Nvidia P100 GPU in an average data center and energy mix consistent with the worldwide average. Some Google data centers already operate on 90% carbon-free energy the overall average was 61% carbon-free energy in 2019 and 67% in 2020.īelow, we illustrate the impact of improving the 4Ms in practice. Google has committed to decarbonizing all energy consumption so that by 2030, it will operate on 100% carbon-free energy, 24 hours a day on the same grid where the energy is consumed. Conventional carbon offsets are usually retrospective up to a year after the carbon emissions and can be purchased anywhere on the same continent. Note that Google matches 100% of its operational energy use with renewable energy sources. These four practices together can reduce energy by 100x and emissions by 1000x. While one might worry that map optimization could lead to the greenest locations quickly reaching maximum capacity, user demand for efficient data centers will result in continued advancement in green data center design and deployment. Moreover, the cloud lets customers pick the location with the cleanest energy, further reducing the gross carbon footprint by 5x–10x. On-premise data centers are often older and smaller and thus cannot amortize the cost of new energy-efficient cooling and power distribution systems. Cloud-based data centers are new, custom-designed warehouses equipped for energy efficiency for 50,000 servers, resulting in very good power usage effectiveness (PUE). Computing in the Cloud rather than on premise reduces energy usage and therefore emissions by 1.4x–2x. Using processors and systems optimized for ML training, versus general-purpose processors, can improve performance and energy efficiency by 2x–5x. Selecting efficient ML model architectures, such as sparse models, can advance ML quality while reducing computation by 3x–10x. We identified four best practices that reduce energy and carbon emissions significantly - we call these the “4Ms” - all of which are being used at Google today and are available to anyone using Google Cloud services. The 4Ms: Best Practices to Reduce Energy and Carbon Footprints We demonstrate four key practices that reduce the carbon (and energy) footprint of ML workloads by large margins, which we have employed to help keep ML under 15% of Google’s total energy use. In “ The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink”, accepted for publication in IEEE Computer, we focus on operational carbon emissions - i.e., the energy cost of operating ML hardware, including data center overheads - from training of natural language processing (NLP) models and investigate best practices that could reduce the carbon footprint. While these assertions rightfully elevated the discussion around carbon emissions in ML, they also highlight the need for accurate data to assess true carbon footprint, which can help identify strategies to mitigate carbon emission in ML.
Machine learning (ML) has become prominent in information technology, which has led some to raise concerns about the associated rise in the costs of computation, primarily the carbon footprint, i.e., total greenhouse gas emissions. Posted by David Patterson, Distinguished Engineer, Google Research, Brain Team
0 kommentar(er)
