Google Announces Video Generation LLM VideoPoet | Canada News Media
Connect with us

Tech

Google Announces Video Generation LLM VideoPoet

Published

9 months ago

 on

Google Research recently published their work on VideoPoet, a large language model (LLM) that can generate video. VideoPoet was trained on 2 trillion tokens of text, audio, image, and video data, and in evaluations by human judges its output was preferred over that of other models.

Unlike many image and video generation AI systems that use diffusion models, VideoPoet uses a Transformer architecture that is trained to handle multiple modalities. The model can handle multiple input and output modalities by using different tokenizers. After training, VideoPoet can perform a variety of zero-shot generative tasks, including text-to-video, image-to-video, video inpainting, and video style transfer. When evaluated on a variety of benchmarks, VideoPoet achieves “competitive” performance compared to state-of-the-art baselines. According to Google,

Through VideoPoet, we have demonstrated LLMs’ highly-competitive video generation quality across a wide variety of tasks, especially in producing interesting and high quality motions within videos. Our results suggest the promising potential of LLMs in the field of video generation. For future directions, our framework should be able to support “any-to-any” generation, e.g., extending to text-to-audio, audio-to-video, and video captioning should be possible, among many others.

Although OpenAI’s ground-breaking DALL-E model was an early example of using Transformers or LLMs to generate images from text prompts, diffusion models such as Imagen and Stable Diffusion soon became the standard architecture for generating images. More recently, researchers have trained diffusion models to generate short videos; for example, Meta’s Emu and Stability AI’s Stable Video Diffusion, which InfoQ covered in 2023.

With VideoPoet, Google returns to the Transformer architecture, citing the advantage of re-using infrastructure and optimizations developed for LLMs. The architecture also supports multiple modalities and tasks, in contrast to diffusion models, which according to Google require “architectural changes and adapter modules” to perform different tasks.

The key to VideoPoet’s support for multiple modalities is a set of tokenizers. The Google team used a video tokenizer called MAGVIT-v2 and an audio tokenizer called SoundStream; for text they used T5‘s pre-trained text embeddings. From there, the model uses a decoder-only autoregressive Transformer model to generate a sequence of tokens, which can then be converted into audio and video streams by the tokenizers.

VideoPoet was trained to perform eight different tasks: unconditioned video generation, text-to-video, video prediction, image-to-video, video inpainting, video stylization, audio-to-video, and video-to-audio. The model was trained on 2 trillion tokens, from a mix of 1 billion image-text pairs and 270 million videos.

The research team also discovered the model exhibited several emergent capabilities by chaining together several operations, for example, VideoPoet can use image-to-video to animate a single image, then apply stylization to apply visual effects. It can also generate long-form video, maintain consistent 3D structure, and apply camera motion from text prompts.

In a Hacker News discussion about VideoPoet, one user wrote:

The results look very impressive. The prompting however, is a bit weird – there’s suspiciously many samples with an “8k”-suffix, presumably to get more photorealistic results? I really don’t like that kind of stuff, when prompting becomes more like reciting sacred incantations instead of actual descriptions of what you want.

The VideoPoet demo site contains several examples of the model’s output, including a one-minute video short-story.

 

Source link

Related Topics:
Continue Reading

News

The Internet is Littered in ‘Educated Guesses’ Without the ‘Education’

Published

1 week ago

 on

October 8, 2024

By

Although no one likes a know-it-all, they dominate the Internet.

The Internet began as a vast repository of information. It quickly became a breeding ground for self-proclaimed experts seeking what most people desire: recognition and money.

Today, anyone with an Internet connection and some typing skills can position themselves, regardless of their education or experience, as a subject matter expert (SME). From relationship advice, career coaching, and health and nutrition tips to citizen journalists practicing pseudo-journalism, the Internet is awash with individuals—Internet talking heads—sharing their “insights,” which are, in large part, essentially educated guesses without the education or experience.

The Internet has become a 24/7/365 sitcom where armchair experts think they’re the star.

Not long ago, years, sometimes decades, of dedicated work and acquiring education in one’s field was once required to be recognized as an expert. The knowledge and opinions of doctors, scientists, historians, et al. were respected due to their education and experience. Today, a social media account and a knack for hyperbole are all it takes to present oneself as an “expert” to achieve Internet fame that can be monetized.

On the Internet, nearly every piece of content is self-serving in some way.

The line between actual expertise and self-professed knowledge has become blurry as an out-of-focus selfie. Inadvertently, social media platforms have created an informal degree program where likes and shares are equivalent to degrees. After reading selective articles, they’ve found via and watching some TikTok videos, a person can post a video claiming they’re an herbal medicine expert. Their new “knowledge,” which their followers will absorb, claims that Panda dung tea—one of the most expensive teas in the world and isn’t what its name implies—cures everything from hypertension to existential crisis. Meanwhile, registered dietitians are shaking their heads, wondering how to compete against all the misinformation their clients are exposed to.

More disturbing are individuals obsessed with evangelizing their beliefs or conspiracy theories. These people write in-depth blog posts, such as Elvis Is Alive and the Moon Landings Were Staged, with links to obscure YouTube videos, websites, social media accounts, and blogs. Regardless of your beliefs, someone or a group on the Internet shares them, thus confirming your beliefs.

Misinformation is the Internet’s currency used to get likes, shares, and engagement; thus, it often spreads like a cosmic joke. Consider the prevalence of clickbait headlines:

  • You Won’t Believe What Taylor Swift Says About Climate Change!
  • This Bedtime Drink Melts Belly Fat While You Sleep!
  • In One Week, I Turned $10 Into $1 Million!

Titles that make outrageous claims are how the content creator gets reads and views, which generates revenue via affiliate marketing, product placement, and pay-per-click (PPC) ads. Clickbait headlines are how you end up watching a TikTok video by a purported nutrition expert adamantly asserting you can lose belly fat while you sleep by drinking, for 14 consecutive days, a concoction of raw eggs, cinnamon, and apple cider vinegar 15 minutes before going to bed.

Our constant search for answers that’ll explain our convoluted world and our desire for shortcuts to success is how Internet talking heads achieve influencer status. Because we tend to seek low-hanging fruits, we listen to those with little experience or knowledge of the topics they discuss yet are astute enough to know what most people want to hear.

There’s a trend, more disturbing than spreading misinformation, that needs to be called out: individuals who’ve never achieved significant wealth or traded stocks giving how-to-make-easy-money advice, the appeal of which is undeniable. Several people I know have lost substantial money by following the “advice” of Internet talking heads.

Anyone on social media claiming to have a foolproof money-making strategy is lying. They wouldn’t be peddling their money-making strategy if they could make easy money.

Successful people tend to be secretive.

Social media companies design their respective algorithms to serve their advertisers—their source of revenue—interest; hence, content from Internet talking heads appears most prominent in your feeds. When a video of a self-professed expert goes viral, likely because it pressed an emotional button, the more people see it, the more engagement it receives, such as likes, shares and comments, creating a cycle akin to a tornado.

Imagine scrolling through your TikTok feed and stumbling upon a “scientist” who claims they can predict the weather using only aluminum foil, copper wire, sea salt and baking soda. You chuckle, but you notice his video got over 7,000 likes, has been shared over 600 times and received over 400 comments. You think to yourself, “Maybe this guy is onto something.” What started as a quest to achieve Internet fame evolved into an Internet-wide belief that weather forecasting can be as easy as DIY crafts.

Since anyone can call themselves “an expert,” you must cultivate critical thinking skills to distinguish genuine expertise from self-professed experts’ self-promoting nonsense. While the absurdity of the Internet can be entertaining, misinformation has serious consequences. The next time you read a headline that sounds too good to be true, it’s probably an Internet talking head making an educated guess; without the education seeking Internet fame, they can monetize.

______________________________________________________________

 

Nick Kossovan, a self-described connoisseur of human psychology, writes about what’s

on his mind from Toronto. You can follow Nick on Twitter and Instagram @NKossovan.

 

Continue Reading

Tech

Tight deadlines on software projects can put safety at risk: survey

Published

2 weeks ago

 on

October 8, 2024

By

 

TORONTO – A new survey says a majority of software engineers and developers feel tight project deadlines can put safety at risk.

Seventy-five per cent of the 1,000 global workers who responded to the survey released Tuesday say pressure to deliver projects on time and on budget could be compromising critical aspects like safety.

The concern is even higher among engineers and developers in North America, with 77 per cent of those surveyed on the continent reporting the urgency of projects could be straining safety.

The study was conducted between July and September by research agency Coleman Parkes and commissioned by BlackBerry Ltd.’s QNX division, which builds connected-car technology.

The results reflect a timeless tug of war engineers and developers grapple with as they balance the need to meet project deadlines with regulations and safety checks that can slow down the process.

Finding that balance is an issue that developers of even the simplest appliances face because of advancements in technology, said John Wall, a senior vice-president at BlackBerry and head of QNX.

“The software is getting more complicated and there is more software whether it’s in a vehicle, robotics, a toaster, you name it… so being able to patch vulnerabilities, to prevent bad actors from doing malicious acts is becoming more and more important,” he said.

The medical, industrial and automotive industries have standardized safety measures and anything they produce undergoes rigorous testing, but that work doesn’t happen overnight. It has to be carried out from the start and then at every step of the development process.

“What makes safety and security difficult is it’s an ongoing thing,” Wall said. “It’s not something where you’ve done it, and you are finished.”

The Waterloo, Ont.-based business found 90 per cent of its survey respondents reported that organizations are prioritizing safety.

However, when asked about why safety may not be a priority for their organization, 46 per cent of those surveyed answered cost pressures and 35 per cent said a lack of resources.

That doesn’t surprise Wall. Delays have become rampant in the development of tech, and in some cases, stand to push back the launch of vehicle lines by two years, he said.

“We have to make sure that people don’t compromise on safety and security to be able to get products out quicker,” he said.

“What we don’t want to see is people cutting corners and creating unsafe situations.”

The survey also took a peek at security breaches, which have hit major companies like London Drugs, Indigo Books & Music, Giant Tiger and Ticketmaster in recent years.

About 40 per cent of the survey’s respondents said they have encountered a security breach in their employer’s operating system. Those breaches resulted in major impacts for 27 per cent of respondents, moderate impacts for 42 per cent and minor impacts for 27 per cent.

“There are vulnerabilities all the time and this is what makes the job very difficult because when you ship the software, presumably the software has no security vulnerabilities, but things get discovered after the fact,” Wall said.

Security issues, he added, have really come to the forefront of the problems developers face, so “really without security, you have no safety.”

This report by The Canadian Press was first published Oct. 8, 2024.

Companies in this story: (TSX:BB)

The Canadian Press. All rights reserved.

Source link

Continue Reading

Tech

Beware of scams during Amazon’s Prime Big Deal Days sales event: cybersecurity firm

Published

2 weeks ago

 on

October 8, 2024

By

 

As online shoppers hunt for bargains offered by Amazon during its annual fall sale this week, cybersecurity researchers are warning Canadians to beware of an influx of scammers posing as the tech giant.

In the 30 days leading up to Amazon’s Prime Big Deal Days, taking place Tuesday and Wednesday, there were more than 1,000 newly registered Amazon-related web domains, according to Check Point Software Technologies, a company that offers cybersecurity solutions.

The company said it deemed 88 per cent of those domains malicious or suspicious, suggesting they could have been set up by scammers to prey on vulnerable consumers. One in every 54 newly created Amazon-related domain included the phrase “Amazon Prime.”

“They’re almost indiscernible from the real Amazon domain,” said Robert Falzon, head of engineering at Check Point in Canada.

“With all these domains registered that look so similar, it’s tricking a lot of people. And that’s the whole intent here.”

Falzon said Check Point Research sees an uptick in attempted scams around big online shopping days throughout the year, including Prime Days.

Scams often come in the form of phishing emails, which are deceptive messages that appear to be from a reputable source in attempt to steal sensitive information.

In this case, he said scammers posing as Amazon commonly offer “outrageous” deals that appear to be associated with Prime Days, in order to trick recipients into clicking on a malicious link.

The cybersecurity firm said it has identified and blocked 100 unique Amazon Prime-themed scam emails targeting organizations and consumers over the past two weeks.

Scammers also target Prime members with unsolicited calls, claiming urgent account issues and requesting payment information.

“It’s like Christmas for them,” said Falzon.

“People expect there to be significant savings on Prime Day, so they’re not shocked that they see something of significant value. Usually, the old adage applies: If it seems too good to be true, it probably is.”

Amazon’s website lists a number of red flags that it recommends customers watch for to identify a potential impersonation scam.

Those include false urgency, requests for personal information, or indications that the sender prefers to complete the purchase outside of the Amazon website or mobile app.

Scammers may also request that customers exclusively pay with gift cards, a claim code or PIN. Any notifications about an order or delivery for an unexpected item should also raise alarm bells, the company says.

“During busy shopping moments, we tend to see a rise in impersonation scams reported by customers,” said Amazon spokeswoman Octavia Roufogalis in a statement.

“We will continue to invest in protecting consumers and educating the public on scam avoidance. We encourage consumers to report suspected scams to us so that we can protect their accounts and refer bad actors to law enforcement to help keep consumers safe.”

Falzon added that these scams are more successful than people might think.

As of June 30, the Canadian Anti-Fraud Centre said there had been $284 million lost to fraud so far this year, affecting 15,941 victims.

But Falzon said many incidents go unreported, as some Canadians who are targeted do not know how or where to flag a scam, or may choose not to out of embarrassment.

Check Point recommends Amazon customers take precautions while shopping on Prime Days, including by checking URLs carefully, creating strong passwords on their accounts, and avoiding personal information being shared such as their birthday or social security number.

The cybersecurity company said consumers should also look for “https” at the beginning of a website URL, which indicates a secure connection, and use credit cards rather than debit cards for online shopping, which offer better protection and less liability if stolen.

This report by The Canadian Press was first published Oct. 8, 2024.

Source link

Continue Reading

Trending

Exit mobile version