빠르고 정확한 개발자 빠르고 정확한 개발자

분류 전체보기

• 주식 각 종목의 가장 최근 결과 가져오기

  postgresql 2025. 6. 21. 16:38

  SELECT DISTINCT ON (i.code) i.code, i.date, i.last_actual_price, i.final_predicted_price, i.price_change, i.price_change_percent, i.test_r2, i.volatility, i.prediction_days, i.last_actual_date, i.train_r2, i.test_mse, i.test_rmse, i.test_mae, c.name as company_name, c.rank, i.model_inf..

  Read More
• Dataset 으로 정의된 case class 의 fields 가져오기

  spark 2025. 6. 10. 21:06

  case class PwlLog( log_type: String, device_type: String, event_type: String)val fields: Array[String] = classOf[PwlLog].getDeclaredFields.map(_.getName)spark.read .... .withColumn("log_type", ...) .select(fields.map(col): _*) .as[PwlLog]

  Read More
• 개발 환경 세팅 & Job 제출

  flink 2025. 5. 5. 18:19

  docker-compose.yml 로 flink cluster 시작version: '2'services: jobmanager: image: flink:1.18.1-scala_2.12-java17 ports: - "8081:8081" command: jobmanager environment: - | FLINK_PROPERTIES= jobmanager.rpc.address: jobmanager volumes: - /mnt/c:/opt/flink/c taskmanager: image: flink:1.18.1-scala_2.12-java17 depends_on: - jobmanager com..

  Read More
• TypeTag, ClassTag

  scala/basic 2025. 4. 22. 21:54

  TypeTagimport scala.reflect.runtime.universe._def printType[T: TypeTag](value: T): Unit = { println(s"Type: ${typeOf[T]}")}printType(42) // 출력: Type: IntprintType("Hello") // 출력: Type: String ClassTagimport scala.reflect.ClassTagdef createArray[T: ClassTag](size: Int): Array[T] = { new Array[T](size)}val intArray = createArray[Int](5) // Int 배열 생성val strArray = createArray[Strin..

  Read More
• Semaphore 를 사용해 최대 maxConcurrent 개의 Future만 실행되도록 제한

  scala/basic 2025. 3. 13. 10:28

  내가 작성package com.naver.search.webimport java.util.concurrent.Semaphoreimport scala.concurrent.ExecutionContext.Implicits.globalimport scala.concurrent.{Await, Future}object SemaphoreFutureTest { def semFuture[T](f: => T, n: Int): Future[T] = { val sem = new Semaphore(n) Future { try { sem.acquire() f } finally { sem.release() } } } def f(x: Int): Int = { Thread.sleep(1000) pr..

  Read More
• Session Kill

  graph database/nebula graph 2025. 3. 10. 20:45

  아래 nGQL 로 session 을 확인 / KILL 할 수 있다. show SESSIONS;kill SESSION   그런데 연결된 session 이 너무 많아 하나씩 KILL 하기 힘들 때는 코드를 작성해 진행할 수 있다.import com.vesoft.nebula.client.graph.SessionPoolimport com.vesoft.nebula.client.graph.data.ResultSetimport scala.collection.JavaConverters._object KillAllSessions { def main(args: Array[String]): Unit = { val sessionPool: SessionPool = ... val resultSet: ResultSet = se..

  Read More
• nGQL Basic

  graph database/nebula graph 2025. 3. 10. 18:16

  https://docs.nebula-graph.io/3.8.0/2.quick-start/4.nebula-graph-crud/Space 생성CREATE SPACE `sites` (partition_num = 1000, replica_factor = 3, vid_type = FIXED_STRING(64)) COMMENT = "site relations" TAG, EDGE 만들기CREATE TAG site(url string);CREATE EDGE follow(weight int); VERTEX, EDGE 데이터 넣기INSERT VERTEX site(url) VALUES "64f3134d1e65568291bac977cf60019a2e1d2e1e537231a88f4ab90c6ab0b9c9":("http://ho..

  Read More
• pandas rolling 을 pyspark window function 으로 표현해 보기

  pyspark 2025. 3. 5. 21:39

  Pandas rollingimport pandas as pddf = pd.read_csv('./UDEMY_TSA_FINAL/Data/starbucks.csv', index_col='Date', parse_dates=True)df.rolling(window=7).mean() Pyspark window functionimport pysparkfrom pyspark.sql import SparkSession, Windowimport pyspark.sql.functions as Fspark = SparkSession.builder.appName('spark_test').master("local[*]").getOrCreate()df = spark.read.csv('./UDEMY_TSA_FINAL/Data/star..

  Read More