org.jsoup.Connection.userAgent()方法的使用及代码示例

x33g5p2x  于2022-01-18 转载在 其他  
字(6.4k)|赞(0)|评价(0)|浏览(210)

本文整理了Java中org.jsoup.Connection.userAgent()方法的一些代码示例，展示了Connection.userAgent()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Connection.userAgent()方法的具体详情如下：
包路径：org.jsoup.Connection
类名称：Connection
方法名：userAgent

Connection.userAgent介绍

[英]Set the request user-agent header.
[中]设置请求用户代理标头。

代码示例

代码示例来源：origin: RipMeApp/ripme

public Http userAgent(String ua)  {
  connection.userAgent(ua);
  return this;
}
public Http retries(int tries) {

代码示例来源：origin: RipMeApp/ripme

private String vscoImageToURL(String url) throws IOException{
  Document page = Jsoup.connect(url).userAgent(USER_AGENT)
                   .get();
  //create Elements filled only with Elements with the "meta" tag.
  Elements metaTags = page.getElementsByTag("meta");
  String result = "";
  for(Element metaTag : metaTags){
    //find URL inside meta-tag with property of "og:image"
    if (metaTag.attr("property").equals("og:image")){
      String givenURL = metaTag.attr("content");
      givenURL = givenURL.replaceAll("\\?h=[0-9]+", "");//replace the "?h=xxx" tag at the end of the URL (where each x is a number)
      
      result = givenURL;
      LOGGER.debug("Found image URL: " + givenURL);
      break;//immediately stop after getting URL (there should only be 1 image to be downloaded)
    }
  }
  
  //Means website changed, things need to be fixed.
  if (result.isEmpty()){
    LOGGER.error("Could not find image URL at: " + url);
  }
  
  return result;
  
}

代码示例来源：origin: RipMeApp/ripme

.header("Referer", this.url.toExternalForm())
.ignoreContentType(true)
.userAgent(USER_AGENT)
.timeout(5000)
.data(postData)

代码示例来源：origin: RipMeApp/ripme

private JSONArray getPageUrls() {
  String postURL = "http://www.tsumino.com/Read/Load";
  try {
    // This sessionId will expire and need to be replaced
    cookies.put("ASP.NET_SessionId","c4rbzccf0dvy3e0cloolmlkq");
    Document doc = Jsoup.connect(postURL).data("q", getAlbumID()).userAgent(USER_AGENT).cookies(cookies).referrer("http://www.tsumino.com/Read/View/" + getAlbumID()).get();
    String jsonInfo = doc.html().replaceAll("<html>","").replaceAll("<head></head>", "").replaceAll("<body>", "").replaceAll("</body>", "")
        .replaceAll("</html>", "").replaceAll("\n", "");
    JSONObject json = new JSONObject(jsonInfo);
    return json.getJSONArray("reader_page_urls");
  } catch (IOException e) {
    LOGGER.info(e);
    sendUpdate(RipStatusMessage.STATUS.DOWNLOAD_ERRORED, "Unable to download album, please compete the captcha at http://www.tsumino.com/Read/Auth/"
        + getAlbumID() + " and try again");
    return null;
  }
}

代码示例来源：origin: RipMeApp/ripme

private String getImageLinkFromDLLink(String url) {
  try {
    Connection.Response response = Jsoup.connect(url)
        .userAgent(USER_AGENT)
        .timeout(10000)
        .cookies(cookies)
        .followRedirects(false)
        .execute();
    String imageURL = response.header("Location");
    LOGGER.info(imageURL);
    return imageURL;
    } catch (IOException e) {
      LOGGER.info("Got error message " + e.getMessage() + " trying to download " + url);
      return null;
    }
}

代码示例来源：origin: ChinaSilence/any-video

public static Document getDocWithPC(String url) {
  try {
    return Jsoup.connect(url).userAgent(UA_PC).timeout(TIME_OUT).ignoreContentType(true).get();
  } catch (IOException e) {
    log.error(ERROR_DESC + url);
    throw new AnyException(ERROR_DESC + url);
  }
}

代码示例来源：origin: RipMeApp/ripme

.userAgent(AbstractRipper.USER_AGENT)
    .get();
for (Element el : doc.select("meta")) {

代码示例来源：origin: RipMeApp/ripme

private static Document getDocument(String strUrl) throws IOException {
  return Jsoup.connect(strUrl)
              .userAgent(USER_AGENT)
              .timeout(10 * 1000)
              .maxBodySize(0)
              .get();
}

代码示例来源：origin: RipMeApp/ripme

LOGGER.info("    Retrieving " + newUrl);
doc = Jsoup.connect(newUrl)
          .userAgent(USER_AGENT)
          .get();

代码示例来源：origin: ChinaSilence/any-video

public static Document getDocWithPhone(String url) {
  try {
    return Jsoup.connect(url).userAgent(UA_PHONE).timeout(TIME_OUT).ignoreContentType(true).validateTLSCertificates(false).get();
  } catch (IOException e) {
    log.error(ERROR_DESC + url);
    throw new AnyException(ERROR_DESC + url);
  }
}

代码示例来源：origin: ChinaSilence/any-video

public static Document getDocWithPhone(String url, String cookie) {
  try {
    return Jsoup.connect(url).userAgent(UA_PHONE).timeout(TIME_OUT).header("Cookie", cookie).ignoreContentType(true).get();
  } catch (IOException e) {
    log.error(ERROR_DESC + url);
    throw new AnyException(ERROR_DESC + url);
  }
}

代码示例来源：origin: ChinaSilence/any-video

private Document requestAPI(String keyword) {
  try {
    return Jsoup.connect(api).userAgent(ua).ignoreContentType(true).data("wd", keyword).get();
  } catch (IOException e) {
    throw new AnyException(ExceptionEnum.VIDEO_SEARCH_ERROR);
  }
}

代码示例来源：origin: RipMeApp/ripme

private void defaultSettings() {
  this.retries = Utils.getConfigInteger("download.retries", 1);
  connection = Jsoup.connect(this.url);
  connection.userAgent(AbstractRipper.USER_AGENT);
  connection.method(Method.GET);
  connection.timeout(TIMEOUT);
  connection.maxBodySize(0);
}

代码示例来源：origin: loklak/loklak_server

meetupHTML = Jsoup.connect(url).userAgent("Mozilla)").get();

代码示例来源：origin: ahmetaa/zemberek-nlp

.userAgent(
  "Mozilla/5.0 (Windows; U; WindowsNT 5.1; en-US; rv1.8.1.6) Gecko/20070725 Firefox/2.0.0.6")
.referrer("http://www.google.com")

代码示例来源：origin: gofore/aws-training

private Document getDocument(String url) {
  try {
    return Jsoup.connect(url).userAgent(USER_AGENT).get();
  } catch (IOException ex) {
    throw new RuntimeException(ex);
  }
}

代码示例来源：origin: malmstein/yahnac

private static Connection defaultConnection(String baseUrlExtension) {
  Connection conn = Jsoup.connect(BASE_URL + baseUrlExtension)
      .timeout(TIMEOUT_MILLIS)
      .userAgent(USER_AGENT);
  conn.header("Accept-Encoding", "gzip");
  return conn;
}

代码示例来源：origin: Fanping/iveely.search

protected Document getDocument(final WebUrl webUrl) {
 try {
  Document document = Jsoup.connect(webUrl.getUrl()).userAgent(this
    .userAgent)
    .timeout(20 * 1000).get();
  return document;
 } catch (IOException e) {
  logger.warn("Get document failed.", webUrl);
 }
 return null;
}

代码示例来源：origin: SudaVideo/MyVideoApi

public static Document getDocWithPhone(String url) {
    try {
      return Jsoup.connect(url).userAgent(UA_PHONE).timeout(TIME_OUT).ignoreContentType(true).validateTLSCertificates(false).get();
    } catch (IOException e) {
      log.error(ERROR_DESC + url);
      throw new BizException(BizErrorCodeConstants.S0002, e);
    }
  }
}

代码示例来源：origin: BeelGroup/Docear-Desktop

protected Connection getConnection(String URL) {		
  return Jsoup.connect(URL)				   
        .ignoreContentType(true)
        .userAgent(this.userAgent)  
        .referrer(this.referrer)   
        .timeout(this.timeout) 
        .followRedirects(this.followRedirects);		           
}

内容来源于网络，如有侵权，请联系作者删除！

Connection

相关文章

    查看更多
    微信公众号

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com