The Trump administration’s efforts to thwart a perceived security threat from video-sharing app TikTok faces challenges beyond those it faced when taking on other Chinese-owned businesses such as Huawei Technologies Co., in part because the U.S. has never blacklisted a wildly popular app.

The millions of Americans who have flocked to TikTok for amusement during the coronavirus lockdown pose just one of several possible hurdles, which is likely leading the Trump administration to move cautiously as it considers taking action, said Samm Sacks, a cybersecurity specialist at Washington-based think tank New America.

“They are trying to figure out how to thread the needle and avoid that backlash,” Ms. Sacks said.

U.S. officials say they are concerned that TikTok, owned by Beijing-based ByteDance Ltd., could pass on the data it collects from Americans streaming videos to China’s authoritarian government. TikTok has said it would never do so. U.S. officials also are increasingly concerned about the risk of misinformation and Chinese propaganda being spread on the app.

The Trump administration has several tools at its disposal to hobble TikTok, including unwinding the cross-border merger that gave TikTok a big user base in the U.S.

Another possibility would be for President Trump to invoke his seldom-used international emergency economic powers to bar social-media apps with ties to foreign adversaries.

The administration also could seek to include TikTok under rules the Commerce Department is developing to ban the use and installation of foreign technology that threatens U.S. national security.

Yet another option would be to put the app on a so-called entity list, which the Commerce Department used to block U.S. companies from selling to Huawei. Unlike TikTok, however, Huawei never had a significant consumer market in the U.S.

Putting TikTok on the entity list—essentially an export black list—could force app providers such as Apple Inc.’s App Store and Google Play to drop TikTok from their offerings, and would likely be challenged, legal and cybersecurity specialists familiar with the issues say.

“Using export regulations may not actually legally apply to an app,” said Theresa Payton, former White House chief information officer under George W. Bush, who now runs a cybersecurity consulting firm. “Typically the U.S. uses it to bar companies from exporting key technology. Not sure they have jurisdiction or legal precedent to extend it to Apple and Google.”

Commerce Department officials didn’t immediately respond to a request for comment.

A TikTok spokesman said the company “won’t comment on specific theoretical options beyond saying that we are committed to ensuring the long-term success of TikTok in the U.S.”

TikTok parent ByteDance has recently considered changing the corporate structure of TikTok as it comes under scrutiny in the U.S. and elsewhere over its Chinese ties. TikTok also recently hired longtime Disney Co. executive Kevin Mayer, who is based in Los Angeles, as its CEO.

Best known for whimsical user-made dance and music videos, TikTok was second only to Zoom for downloads in the U.S. in the first half of 2020, according to market research firm Sensor Tower. The app has been downloaded more than 180 million times to date in the U.S. through Apple’s App Store and Google Play.

Any action against TikTok would likely further fray strained relations with China, but U.S. officials say Chinese-owned apps such as TikTok and Tencent Holdings Ltd. ’s multipurpose WeChat pose a risk through the data they soak up on American users.

“There are a number of administration officials who are looking at the national security risk as it relates to TikTok, WeChat and other apps that have the potential for national security exposure, specifically as it relates to the gathering of information on American citizens by a foreign adversary,” White House chief of staff Mark Meadows told reporters recently. “I don’t think there’s any self-imposed deadline for action, but I think we are looking at weeks, not months.”

That is cheering some TikTok critics in Congress.

“I’m heartened by the tough tone” coming from the administration lately, said Sen. Josh Hawley (R., Mo.), who has introduced several bills taking aim at TikTok, including one to prevent federal employees from using it. “This isn’t going to stop” without concerted government action, he said, referring to apps tied to foreign adversaries.

Adding to the concerns is the looming election season. Some lawmakers worry that TikTok could become a haven for political misinformation and Chinese propaganda.

Mr. Trump’s re-election campaign, meanwhile, has run recent ads on Facebook accusing TikTok of spying on its users. “TikTok has been caught red-handed by monitoring what is on your phone’s clipboard,” one ad said.

“We don’t want to be in the business of casually stripping away Americans’ rights to use apps they like, but we must take into account when a foreign authoritarian regime that suppresses free speech for its citizens uses malign and covert means to influence speech in free countries,” said Rep. Michael McCaul of Texas, the top Republican on the House Foreign Affairs Committee, in a statement.

Ms. Payton, who wrote “Manipulated: Inside the Cyberwar to Hijack Elections and Distort the Truth,” said TikTok’s young audience can be especially vulnerable to such misinformation.

“It could be a potential disaster for our presidential election cycle,” said Ms. Payton. “We used to socialize on our neighborhood square. Now with the pandemic this is our neighborhood square.”

Some are cautioning against the administration’s approach, however.

“TikTok is a potential security menace, but banning TikTok hardly confronts the profound threat China poses to our national security, economy, & democracy,” Sen. Richard Blumenthal (D., Conn.) tweeted recently.

Secretary of State Mike Pompeo, however, said on Fox News earlier this month that the administration is taking the perceived threat from TikTok seriously.

“With respect to Chinese apps on people’s cellphones, I can assure you the United States will get this one right,” he said.